Operational dashboarding and reporting with Microsoft Business Intelligence Solutions

Editor's Notes

• #2: .
• #8: A dashboard is a key piece of equipment. There’s one in this Formula car – even though there’s no dash. Driver is interested in a few things. In order, they are his location, his speed, his gear, and the condition of the critical parts of his vehicle From the drivers viewport the “UI” prioritizes these items in order of importance At a glance, these things can be discerned But dashboards mean different things to different people
• #9: This is an old one from an old car, and there’s nothing wrong with it. At a glance, you can determine the speed of the vehicle, how much fuel is available, and if there is something wrong. This is likely about the oldest example of a KPI that I’ve ever seen as well
• #10: This dashboard is a little different Granted, on this airplane, there are many more parameters to measure than on the old car. All of that information is available, but it’s hard to spot It’s not hard to spot if you have a trained eye Pilots need immediate access to aircraft conditions, hence this complicated setup, but they are trained. It’s not complicated to them. Dashboards have different design points for different audiences.
• #11: Here’s a newer one from a newer car Speed is still the most important thing, but fuel is only important when there isn’t enough, so there is an alert. The amount of fuel can still be seen, as can RPM, or optionally through configuration, a series of other parameters that the driver may be interested in. We can also show other interesting information customized to the driver – in this case, the song that is currently playing. The points are that - we are monitoring the status of something - We are (or should be) limited by the operator’s ability to absorb information or by the user interface - We may be limited by the available data
• #12: Open the Dashboard Site Overview Dashboard Note the different measures Open a few of the KPIs Open GDP and KPI for Americas Select Canada and note how the 3 graphs update Select Financial Dashboard and select Canada Next, select China Note the increase in GDP per Capita, and poverty Open health dashboard. Select China - note the change in Child Mortality Open education dashboard - use Europe data Note that these were all built with different tools, and they all have different strengths. We will show how it's been done..
• #13: There are many products that make up the BI stack and many of them will be used one way or another in any dashboard project. What’s important is to understand what they are, and how they fit. In some cases, there are multiple ways to achieve the same goal, so the implications of using any should be understood. The good news is that these all fit together into a few product groups. We’ll got through these today.
• #14: Keep in mind that in every case, SharePoint is the preferred publishing platform SSRS is the only exception – it can be hosted on its own, but significant advantages are there for SharePoint These approaches are not mutually exclusive, and in some cases can’t be. For example, SharePoint Mashups will depend on content from any or all of Excel, SSRS, PerformancePoint, or other SharePoint content. We’re going to walk through all of these today with the exception of Azure VMs. Azure VMs take the same approach as on-prem, they’re just hosted in the cloud – the “private cloud” No matter which approach we take, they all have the same basic underpinnings. These are based on fundamental BI concepts that need to be understood. So let’s dive in. First and foremost, there is one fundamental Business Intelligence rule that must be adhered to.
• #15: Business Intelligence is all about the data, but that doesn’t mean that you just wire up Excel to source data and start Extracting (although far too many people do). This is bad for a number of reason - Security – data level access to production data - Usability – difficult to understand constructs (Great Plains anyone?) - Performance – reporting against the production data concentrates the load. - Organization – data optimized for transactions, not reporting
• #16: Instead of querying our source systems directly, we want to take our data and move it into Data Warehouses and data marts, which are optimized for the sorts of analysis that we want to perform. This is done through an ETL operation. CLICK The data is extracted from the source system, CLICK transformed into the shape we need it, CLICK then loaded into the data warehouse. CLICK Other ETL processes or cube process will load the data into any necessary marts, cubes or models. From here various servers and client will access the data, usually from the data marts of cubes, but occasionally from the warehouse directly. So how does this translate to the Microsoft stack? There are two ways. The Enterprise, or “classic” BI method, or the Power (personal) method.
• #17: Starting with the classic method, SQL Server Integration Services is the tool that performs our ETL. SQL Server Database Engine is used for the storage of the data warehouses and data marts SQL Server Analysis Services is the multidimensional engine (traditional OLAP cubes) and now is the engine for enterprise tabular models (xVelocity). SSRS is the traditional server engine for serving reports, and can be deployed either standalone, or through SharePoint. These tools all ship on SQL server media, but some (SSRS and PowerPivot for SharePoint) may be deployed to SharePoint Clients of this infrastructure may be servers themselves, or designers and Power Users. Consuming tools include Excel, SQL Server Data Tools, Excel Services, PowerPivot for SharePoint, or a host of other tools. Recently, there has been a lot of work in the Personal BI space – so how does that compare to this approach? Fundamental BI concepts still apply.
• #18: To start with, we have an Excel Workbook. Excel is the personal BI client from Microsoft. As of the 2010 version (through an add-in), or Excel 2013 directly we have access to an embedded xVelocity data model. CLICK Using the PowerPivot add-in (needs to be enabled) we can import data directly from the source data systems, and then manipulate the structure, but the data is read only. It can be refreshed, but not edited. Really, we have the E and the L of an ETL system. CLICK More recently, Power Query has been introduced. It’s a part of Power BI, but in this context it’s just a free Excel add-in that brings more elegance to the import. It puts the T back into ETL on the personal side. It has a host of other features, and different data source options, but that’s fundamentally what it is. Power Query can also load data directly into the workbook, into the model, or both. CLICK Once the data has been loaded it can be consumed through a number of Excel tools. The traditional multi dimensional tools are the Pivot chart and Pivot table, but we now also have Power View for analytical reporting, and Power Map for geospatial analysis. PowerPivot is the model editor. Be careful with Power Query. It cant be automatically refreshed. Yet. All of these approaches, both enterprise and personal converge through SharePoint and dashboards.
• #19: Within SharePoint, we can publish reports and data models, and establish connections to the relevant back end systems. These components can then be used to construct dashboards, or used on their own as dashboards. Dashboards can contain, but are not necessarily limited to Worksheets and worksheet components through Excel Services, either directly connected or via PowerPivot SSRS Reports PerformancePoint scorecards and KPIs PerformancePoint reports
• #24: There are different tools for different tasks, and SharePoint is no exception. Given that we can mashup resources from a number of different products, its no surprise that we need to use several tools to accomplish it. We’re going to have a quick look at most of these in turn, and we’ll start with Reporting Services
• #27: There are lots of things to worry about… this is what comes up frequently that doesn’t fit into a technical demo
• #28: - People sometimes get hung up on per user authentication - “You can’t do BI without Kerberos” – which is wrong - In many cases, this is simply not true. Service accounts can impersonate end users down the chain. SSRS has SetUser(), Excel Services and PerformancePoint support EffectiveuserName against multidimensional BISM supports impersonation in SSRS Power View No per User auth of any sort with PowerBI – control through document security
• #30: - Depending on the tool, data freshness may be an issue Latency between data source and data mart Latency between data mart and data models What is “Real enough” time? There is a trade off between freshness and cost There are BI Tools for Real Time
• #31: Data is like food. Not everything needs to be fresh…. SSRS, which we use to power the ETL, uses SQL ‘s agent jobs to run on a schedule. These are quite granular, down to the minute level. Any tools leveraging the data marts/cubes directly can be that fresh. However, constant runs are resource intensive PowerPivot for SharePoint can be scheduled for refresh daily. This is also true for refreshes in Power BI using the Data Management Gateway, although both can be refreshed on demand. There is a hack for PP4 SP If Real Time analysis is actually necessary, there are tools available. StreamInsight for ETL, and DirectQuery for tabular models.
• #33: So what do we need to make all of this work?
• #34: Just because it comes from SQL doesn’t mean it belongs there. Power BI is an odd duck. It’s from 2 quadrants.
• #36: Reporting Services is the only one that works well with Standard Edition SharePoint Excel and PerformancePoint require SharePoint Enterprise PowerPivot requires BI Edition or Enterprise SQL Server Power BI is an additional Office 365 licence on top of E3
• #38: - What does the future hold? Almost all development efforts are going into the Power stack Reporting Services still fills and important niche, and is relatively complete PerformancePoint? If Excel gets features, it’s redundant I like the model deployment from Excel to SSAS. Is Power Query far behind?