Sam Lester - SQL Server & BI Premier Field Engineer

Microsoft Flow – Copy files from SharePoint to a local PC

Samuel Lester - MSFT — Sun, 14 Oct 2018 17:54:12 GMT

Microsoft Flow – Copy files from SharePoint Online to a local PC

I recently assisted a customer with an automated workflow through Microsoft Flow that copies files from a SharePoint Online site library to a file system share on a local machine being used as an archive server. I found several forum discussions and blog posts with a similar workflow, but nothing that helped solve the issues I was hitting. As a result, the goal of this post is to walk through this scenario including screenshots and descriptions of where I was getting stuck. Log in to Flow (https://us.flow.microsoft.com). In the top right, click on the settings/gear icon and navigate to the Connections site. We will be creating a connection in Office 365 that leverages the on-premises data gateway. This connection will then later be used in our Flow scenario to establish the connection to the file share from Office 365 (or Azure). In the Connections screen, click on the New Connection button to add a new connection. In the search bar on the top right, enter “File System” and hit the search button. This will allow us to add the file system connection where you’ll see the following screenshot. Enter the details for the root folder in the form of a local drive (c:\test) or a network share (\\server1\folder) and supply credentials to access this location. After supplying this information, you need to supply the valid on-premises data gateway, which acts as a bridge between on-premises data and Azure cloud services, such as Microsoft Flow, PowerApps, Power BI, and Logic Apps. If you haven’t yet set up the on-prem data gateway, it is very quick and straightforward following the documentation. Now that we have the connection established, we will create the Flow. There are several methods to create a new Flow leveraging existing templates. In my scenario, I created it within the SharePoint library. By initiating from this location, we see context-based suggestions based on the SharePoint items. Select the template titled “When a new file is added in SharePoint, complete a custom action”. When you open the new Flow template, you will see a single task titled “When a file is created (properties only)”. One important thing to notice here is that the task only collects the metadata for the entry as opposed to the deprecated version which collected the actual file. This leads to one additional step to get the file contents. In this initial task, you can either provide the site address, library name, and folder where the files will be created or you can also set up another connection through the on-prem data gateway with this information that you can use in this step. Next, we’ll add the step to get the file contents. Click the + button under the existing step and search for “Get file content” in the action search window. Select the highlighted step below to get file contents from SharePoint. In this task, provide the SharePoint site address and select the file Identifier. Now that we have the file contents, the final step is to create the task to copy the file. Add a new step and select the File System task to create a file. Since you have already created the connection through the on-prem data gateway, this step is very easy to configure. Select the folder path from the folder icon. This will reference the path you created in the gateway connection. For the file name and contents, select the dynamic content from the previous steps to carry forward the name and contents. Notice here that you could modify the name for archival purposes if needed. With that, you’ll have a 3-step flow that is initiated with the creation of an entry into a SharePoint library that copies the file to a local file share. Note that there are several very helpful templates in Flow that allow you to automate many other actions. This is a simple example to help get you started. Enjoy, Sam Lester (MSFT)

Power BI Troubleshooting Checklist

Samuel Lester - MSFT — Thu, 15 Mar 2018 18:22:18 GMT

If you have ever started a sentence with “I’m seeing a strange issue in my Power BI Desktop report” and wondered how to investigate, this article is for you.

Over the past few years of troubleshooting Power BI Desktop customer issues, I’ve created a checklist that often helps to identify a root cause within a couple minutes. If there are other handy tricks that you use, please share them in the comments below. Since I work with several customers, most of the requests come via email or social media. Here are the investigation steps/questions that I use:

What version of Power BI Desktop are you using? Does the report include custom visuals or preview features?
- Reason: With the technology improving so quickly, we encounter older reports that have been working for months, then stop working somewhat unexpectedly. These are typically a version problem, either with something that has been updated in the service causing a mismatch with Desktop components or with the existence of older custom visuals that are out of date and causing the issue.
Are you seeing a “Send a frown” pop-up?
- Reason: If so, this is very helpful in that the diagnostics have been aggregated for quick inspection.
Is it failing consistently? Can you reproduce it? Is it something that can be shared with me to investigate?
- Reason: Intermittent issues can be really challenging to investigate, but consistent issues are typically much easier to resolve. Also, getting access to the report makes it significantly easier to troubleshoot. This is obviously not always an option with PII, HBI data, etc., but when available makes it much quicker.
Does it fail when someone else accesses the same report? Does it fail if you open it on a different machine?
- Reason: This is the Power BI version of the question ‘Have you tried rebooting’ to fix an IT problem. By trying it with other users and on other machines, you can narrow down if the problem is a machine issue, user issue, or something else. An extension of this question is to ask if they have tried to rebuild the model from scratch in another instantiation of Power BI Desktop (recreate the report).
Are you using preview features?
- Reason: Preview features are available to use and explore, with the disclaimer that they might be removed in future versions. I have had a couple issues with crashing Power BI Desktop when using preview features, but typically I haven’t seen many issues in this category.
Can you enable diagnostics prior to reproducing the issue and check the logs for any error messages?
- Reason: In the Options menu under the Diagnostics pane, you can enable tracing. The log is quite verbose and helpful. The log is stored under %UserProfile%\Microsoft\Power BI Desktop Store App\Traces. Tracing has identified several issues for me where the other techniques did not reveal any hints.

The previous questions assume that the file can be opened. However, I’ve seen several situations where the .pbix file would not open. In this case, my suggestion is to create a copy of the file, rename the extension to .zip, and unzip the contents. Gilbert Quevauvilliers wrote a great blog exploring these contents to see what is included in the unzipped .pbix file. By unpacking the contents of the .pbix file, I’ve discovered issues where email filters stripped out critical portions of the file and when environment settings prevented the file from being opened. This can also be used to find the versions of the custom visuals included in the report by opening the JSON file under ‘\Report\CustomVisualizations’. If the custom visual has been updated with a newer release, try replacing the existing visual with the updated visual. This has also solved many customer issues. While this is certainly not an exhaustive list, this set of troubleshooting steps has helped me (and now many of my customers and colleagues) get to the root cause of an issue in Power BI. Often, they can be fixed with updating versions, checking the diagnostic logs, or investigating the unpacked contents. In some cases, we’ve ended up with issues that I’ve taken back to the product team that have now been fixed in the current version. If you have additional troubleshooting tips and tricks, please share them in the comments. Thanks, Sam Lester (MSFT)

Importing a .dbf (dBase) file into Power BI

Samuel Lester - MSFT — Fri, 23 Feb 2018 17:46:24 GMT

How can I import data from a .dbf (dBase) file into Power BI?

Today’s Power BI Mailbag Question comes from a friend who often works with geospatial data and, as a result, ends up with several different shapefile formats, including .dbf (dBase) database table files. There are several blog articles around shapefile mapping options in Power BI, but this question of accessing the .dbf file directly from Power BI was something I hadn’t been asked before. Note: Power BI does not have native .dbf file support at the time of publishing this article, but the idea has been proposed at ideas.powerbi.com if you would like to vote for it. When I began searching for a solution, I found several threads in the .NET provider MSDN forum showing how to leverage the Access OLEDB provider (Microsoft.ACE.OLEDB.12.0) to import data from a .dbf file into SQL Server. With this, we can use a similar solution in Power BI. Data Source (Rainfall in Ethiopia): For this example, I’m using a .dbf file containing the rain statistics in towns in Ethiopia from worldbank.org. Steps: In Power BI Desktop, click on Get Data -> Blank Query Click on the Advanced Editor and replace the default text with the following text (replacing with your path and dBase file name): let Source = OleDb.DataSource("Provider=Microsoft.ACE.OLEDB.12.0;Data Source=C:\Blog\DBF\;extended properties=dBASE IV", [Query="select * from [rain.dbf]"]) in Source This will prompt you to edit the credentials. Click "Edit Permissions" to continue. Another prompt appears asking to approve of running the native query against this data source. Since the OLEDB provider syntax we provided contains the query string, you need to approve that this can be executed against the .dbf file. Click Run to continue. You may get one additional dialog box requesting proper credentials as I did with some other data sources I used in testing this solution. I selected the “Default or Custom” option with no value supplied in the connection string property dialog. With this, the data will be loaded into Power BI and you can continue with your report creation. Special thanks to Aaron Bator for constantly pushing the edges of Power BI and bringing new and innovative ideas and questions. Keep them coming! Thanks, Sam Lester (MSFT)

Understanding the differences between “Power BI Desktop” and “Power BI Desktop Optimized for Power BI Report Server”

Samuel Lester - MSFT — Mon, 19 Feb 2018 12:28:38 GMT

Over the past few weeks, I’ve been spending a lot of time discussing Power BI Report Server with my customers and colleagues. One of the most common questions I’m asked is:

What are the differences between “Power BI Desktop” and “Power BI Desktop optimized for Power BI Report Server”?

In this blog post, I’ll cover the similarities and differences between Power BI Desktop and Power BI Desktop Optimized for Power BI Report Server (shortened to “Desktop Optimized” for the sake of this article), including the following topics:

Why are there two versions?
Where do you find the installation media?
Are the installation steps the same?
Key application differences within the splash screen, options menu, and product functionality

Why are there two versions?

With Power BI Desktop, the intended publishing location is the Azure-hosted Power BI Service, which is updated frequently by Microsoft. From this perspective, you can continually upgrade to the monthly Power BI Desktop release and be sure that the reports will work when published to the Power BI Service because updates are being made on both sides. This model changes with the introduction of Power BI Report Server, where the report server hosting the Power BI reports now resides within your environment. As a result, the server version (PBI Report Server) and the client version (Power BI Desktop optimized for PBI Report Server) need to be controlled a bit tighter to ensure compatibility. The published plan is to release a new version of the server and corresponding Desktop Optimized version roughly every 4 months.

Where do you find the installation media?

Power BI Desktop can be found here: https://powerbi.microsoft.com/en-us/desktop/ The executable name is PBIDesktop.msi (for 32-bit installation) or PBIDesktop_X64.msi (for 64-bit installation). Power BI Desktop Optimized for Power BI Report Server can be found under the "Advanced Download Options" link here: https://powerbi.microsoft.com/en-us/report-server/ The executable name is PBIDesktopRS.msi (for 32-bit installation) or PBIDesktopRS_X64.msi (for 64-bit installation).

Installation Process

During the installation process, there are a few small differences between the versions. Power BI Desktop does not contain the PBI RS message, while the Desktop Optimized version contains an additional information message on the setup page. The default installation path is different for the two versions, with Power BI Desktop defaulting to %PROGRAMFILES%\Microsoft Power BI Desktop\, while Desktop Optimized defaults to %PROGRAMFILES%\Microsoft Power BI Desktop RS\.

Splash Screen

When you launch the two applications, you’ll notice a small, but important, difference on the splash screen, where the Desktop Optimized version allows you to “Open from report server”. By selecting this, you provide the connection information to the Power BI Report Server, where you are then shown the list of reports that you have access to on the server. You can open those directly in PBI Optimized and then publish them back up to the PBI Report Server when finished.

Options Menu

The Power BI options menu is an area of the product where you can make optional changes, such as enabling tracing for troubleshooting, enabling preview features, updating security and privacy settings, etc. By comparing the options menus from the two versions, we see that the Power BI Desktop Optimized version does not include full feature parity. The biggest difference (that I noticed) is that Desktop Optimized does not allow for the use of preview features. My hunch for this goes back to the requirement of keeping the Report Server and the Desktop Optimized version in lock-step for compatibility/stability purposes, while the preview features are in the earlier stage of development and testing until they get moved out of preview and into the standard product.

Additional Notes

Power BI Desktop Optimized for Power BI Report Server adds a new option to the “Save as” menu, where you can save directly to the Report Server. The two versions can be installed on the same machine, with the startup menu labels being slightly different. One additional item of note is that custom visuals are supported in the Power BI Desktop Optimized for Power BI Report Server version, with the same methods of adding the visuals (by web or by file share). There are likely several other differences that I haven’t encountered yet. If you have discovered any differences not listed here, please share in the comments below and I will update the article. Thanks, Sam Lester (MSFT)

Help! I can’t close the Power BI Desktop Splash Screen!

Samuel Lester - MSFT — Fri, 16 Feb 2018 22:23:09 GMT

Today’s blog entry is a quick tip to get around what appears to be a Power BI Desktop splash screen that won’t go away. This is the first time I've experienced the issue and since the window is modal and I couldn't find a keyboard sequence to close it, I was stuck until I realized the solution. (Note: I discovered it in Power BI Desktop optimized for Power BI Report Server). Here’s the view that I encountered: In the image, the splash screen seems to be shifted up, which makes the X (close button) almost invisible. When this window is open, clicking anywhere else in the application has no effect since the window is modal. I tried to close the window with various key sequences (Ctrl+X, Ctrl-C, Esc, etc.), but nothing worked. Fortunately, there a few pixels of the close button that are still in view, but it took some time to find it. Since I was documenting scenarios for my customers, I had a screenshot of the initial splash screen, which was as expected. The problem seemed to occur when I opened the application with the auto-recovery toolbar, which was when I noticed the issue for the first time. However, I was surprised to see it remain after I’d removed the auto-recovery files and reopened the application. I've sent the details on to the wonderful Power BI Desktop product team and will update this article once I hear back from their investigation. If you have encountered this issue, search for the small area in the top right of the screen where you can close the window, and please leave a quick comment below. Thanks, Sam Lester (MSFT)

Troubleshooting a Power BI Dataset Refresh Error

Samuel Lester - MSFT — Wed, 07 Feb 2018 21:02:42 GMT

Troubleshooting a Power BI Dataset Refresh Error

I investigated a Power BI Dataset Refresh issue tonight that had been successfully refreshing until earlier in the day, when it began to fail consistently. The report solution is an extension of the Power BI Bing News Solution Template (which is really impressive!). In the Power BI tenant, I navigated to the dataset properties to view the refresh history where I saw the following error message stating that "There was an error when processing the data in the dataset", followed by the additional details of "The command has been canceled". The error message included a table name, but searching through the previous failure messages, the table name was not consistent, so this seemed to be a bit misleading. The message stating that the command had been canceled reminded me of the error in Power BI Desktop when an error is encountered while refreshing a report with multiple datasets. In that case, the refresh fails on the specific problematic data source, with the other sources showing the message below: My next troubleshooting step was to open the report in Power BI Desktop and manually refresh the data sources, which ended up leading to the root cause of the problem. When I manually refreshed my report, I hit the following error message, which highlighted a constraint violation with the data that was being refreshed, in this case where an attempt to insert duplicate values were causing the violation. I manually cleaned up the duplicate entry in the database and the refresh started to succeed again. This one turned out to be a surprisingly quick investigation with Power BI Desktop providing a much more detailed message that led me to the issue. If you encounter a dataset refresh issue, try to refresh the report manually in Power BI Desktop for potentially more information to help troubleshoot. Thanks, Sam Lester (MSFT) PS: Did anyone else notice that the two different error messages spell canceled/cancelled differently? "The command has been canceled." and "Load was cancelled by an error in loading a previous table."

2017 Year in Review - Blog Stats

Samuel Lester - MSFT — Tue, 30 Jan 2018 14:35:53 GMT

Sam Lester’s 2017 Blog Year in Review

2017 was an exciting year for me as I continued to work extensively with Power BI and share my solutions, work with the Microsoft product development team to get several customer bugs fixed in our products and completed the MPP in Big Data. Along the way, I shared a lot of this information through my blog and recently looked back through my blog metrics for the previous year. Here are some fun stats: · 30 new blog posts published · 201,933 page views · 108,897 visitors from 192 countries · 105 comments posted · The top 5 visiting countries are United States (32%), United Kingdom (8.7%), India (8.5%), Germany (5.9%), and Canada (3.7%) · The top 5 visiting cities are London (3%), Bengaluru (2%), Hyderabad (1.4%), Sydney (1.3%), and New York (1.2%) · 13,027 words written

The top 5 articles published in 2017 were:

1. IP Address Mapping in Power BI 2. Calculating the difference between two dates in YEARS, MONTHS, and DAYS in Power BI or Excel 3. Sorting by Month Names in Power BI 4. Power BI Terms and Definitions 5. Comparing Table Styles in Power BI

The top 10 articles by page views in 2017 were:

1. Power BI and SSRS 2016 in an On-Premises Environment 2. So long AdventureWorks, we’ll miss you! (And hello Wide World Importers!) 3. Power BI – 12 Days of Dashboards 4. My Experience Completing the Microsoft Professional Program in Data Science 5. Finding the Correct Version of the AdventureWorks SQL Server Sample Database 6. IP Address Mapping in Power BI 7. EOMONTH() Equivalent in SQL Server 2008 R2 and below 8. Calculating the difference between two dates in YEARS, MONTHS, and DAYS in Power BI or Excel 9. Sorting by Month Names in Power BI 10. TSQL – Solve it YOUR Way – Finding a Person’s Current Age Based on Birth Date

Lifetime Blogging Milestones:

I started blogging on MSDN in 2012 when I was a member of the SQL Server product team. After joining Microsoft Premier Support in 2013, I spent a few years ramping up on customer sites and learning the Premier industry for the next couple years. Since 2015, I have been actively blogging as much as I can (given my current schedule) and look forward to continuing to create content in a learning environment throughout 2018 and beyond. Thank you to everyone who has commented on these articles and shared them over social media. Sam Lester (MSFT)

Exploring Big Data: Course 10 – Microsoft Professional Capstone: Big Data

Samuel Lester - MSFT — Mon, 01 Jan 2018 13:06:18 GMT

(This is course #10 of my review of the Microsoft Professional Program in Big Data)

Course #10 of 10 – Microsoft Professional Capstone: Big Data

Overview: Course #10 brings the Microsoft Professional Program in Big Data to a close. The capstone project is comprised of three different sections, each making up 33% of the overall grade even though the complexity and number of questions varies by section. The project is based on one years’ worth of weekly sales data for roughly 300 stores making the dataset a collection of over 15,000 files. As a result, we need to rely on the techniques learned in previous courses to set up blob storage, use Azure Storage Explorer to copy/store files, author U-SQL jobs to process files, set up the data warehouse using the supplied T-SQL script, create linked services to establish the dataset connections and pipelines, and finally load the data into the data warehouse to query for the final answers. Time Spent / Level of Effort: When I read the first lab exercise instructions, I was a bit nervous about tackling this project. Over the previous nine courses, we’ve covered a LOT of material, much of which is challenging to keep straight. I had completed the 9^th course about two weeks before the capstone opened, so I went back to the previous archived capstone and spent about 12 hours going through the old capstone to practice. As it turned out, this was time very well spent. In total I spent roughly 15 hours for this capstone project. Course Highlight: The highlight of the capstone course was going from reading the original instructions with a hefty dose of confusion to ultimately completing the labs and projects to wrap up the class. It required me going back through several of the previous courses and reading the lab instructions and watching some videos for a second and third time. Suggestions: My biggest suggestion for completing the capstone course is to view the previous capstone course to better understand the tasks. For example, I officially completed the capstone course that opened on January 1, 2018, but I spent several hours working on the previous capstone (October 2017). The time I spent on the October course was extremely helpful when the new course opened since I had already completed most of the exercises. An additional suggestion would be to focus on obtaining the required passing score of 70% prior to the second part of lab 3, where all 15,000+ files need to be loaded and queried. I found this part of the course to be the most challenging, but since I had already scored above 70%, I wasn’t as concerned about getting these questions correct. However, if I’d been below the required 70% at this point, there would have been a lot more pressure to get these questions correct. Finally, use the tips supplied in the lab exercise notes that inform you of the courses where the material was originally introduced. As mentioned above, much of the material overlaps a bit, so knowing which course to revisit saved a lot of time. Once I went back to the specific courses for review, I found the lab instructions for those courses to be extremely helpful. Summary: The 10 courses that make up the Microsoft Professional Program in Big Data are an outstanding way to improve your knowledge about the concepts of processing Big Data. Prior to completing this MPP, several of these topics were vaguely familiar to me, but not well enough to teach/explain to others. After going through this program, I have a much better understanding and will continue to work with these technologies. I hope this blog series has helped you on your journey to improve your Big Data skills. It certainly has helped mine! Thanks, Sam Lester (MSFT)

SQL Server DBA Morning Health Checks

Samuel Lester - MSFT — Fri, 29 Dec 2017 22:03:48 GMT

Introduction: As a Microsoft Premier Field Engineer, I get to work with amazing colleagues who create incredible customer solutions. Patrick Keisler ( blog | | ) is a long time SQL Server professional, who also works as a PFE supporting customers throughout Europe. He recently created a very handy "SQL Server Morning Health Check" solution on short notice for one of his customers. The solution is very helpful and can be added easily to your morning routine as a DBA to monitor your SQL Server environment. Patrick, thanks for sharing!

SQL Server Morning Health Checks (Created by Patrick Keisler)

A few weeks ago, a customer asked me to help develop a way to improve their morning routine of checking the health of each SQL Server. This daily task would typically take about an hour for the DBA to complete. The solution I provided him, reduced that task to under one minute. The DBA needed me to answer these questions for each SQL Server: 1. What is the uptime of each SQL Server? 2. What is the status of each database? 3. What is the status of each Availability Group? 4. What is the backup status of each database? 5. What is the available disk space? 6. Are there any SQL Agent failed jobs in the last 24 hours? 7. What errors appeared in the SQL errorlog in the last 24 hours? In addition, the customer asked to display the results using the typical stop light colors (red, yellow, and green) to represent the condition of each server. You might be thinking this sounds like a dashboard; however, they just wanted something simple that could be run by the DBA each morning, but also something that could be run by the other non-DBAs that provide backup during off-hours. I only had a few days to get a solution up and running, so I needed something quick. To make my task a bit easier, the customer had a Central Management Server (CMS) with all servers registered. Armed with this information, I proceeded to use PowerShell to automate each of these tasks. Step one was to loop through each registered server within CMS. Luckily for me, I had previously written a CMS function (Get-CmsServer) for the SQLPSX module. This function was originally created by Chrissy LeMaire but I had heavily modified it for the SQLPSX project. https://github.com/MikeShepard/SQLPSX https://blog.netnerds.net/smo-recipes/central-management-server/ To use this function, you just need to provide the name of the CMS server and the CMS group to loop through. The output is an array of SQL Servers that you can loop through. $targetServerList = Get-CmsServer –cmsServer ‘SOLO\CMS’ –cmsGroup ‘PRODDUCTION’ In the example above, we’ll connect to the CMS server ‘SOLO\CMS’ and get all the SQL Servers that are registered in the ‘PRODUCTION’ folder and all subfolders that may exist beneath it. In the picture above, if you only wanted to loop through the two servers in the SQL2012 folder, then you would specify the CMS group as -cmsGroup ‘PRODUCTION\SQL2012’. Now that we can get the list of SQL Servers, we need to use SQL Server Management Objects (SMO) to complete the other tasks. The SMO is a .NET component that allows you to perform any management task against a SQL Server. Going into the how of using SMO is beyond the scope of this article, but links at the end of this article will help guide you. For our purposes, we’ll just be creating a simple ServerConnection to SQL Server. $srv = New-Object (‘Microsoft.SqlServer.Management.Common.ServerConnection’) ‘MySqlServer\Instance’ $srv.Connect() In the example above, we create a new ServerConnection object and then call the Connect method. From here, it’s a simple as passing TSQL queries and executing them. $cmd = ‘SELECT sqlserver_start_time FROM sys.dm_os_sys_info;’ $results = $srv.ExecuteWithResults($cmd) The output is stored in a dataset ($sqlStartupTime) which we can then use to calculate the uptime of the SQL Server by using New-TimeSpan. $upTime = New-TimeSpan –Start ($results.Tables[0].sqlserver_start_time) –End ($results.Tables[0].current_timestamp) Now that we have our up-time value, we can proceed with determining the condition level (critical, warning, or good) and then displaying the results. Each one of the condition checks are based on what my customer requested, but they can easily be changed for your needs. Critical = SQL uptime < 6 hours Warning = SQL uptime < 1 day but >= 6 hours Good = SQL uptime > 1 day We can just use a simple if statement to determine the condition. if ($upTime.Day –eq 0 –and $upTime.Hours –lt 6) { #critical } elseif ($upTime.Day –lt 1 –and $upTime.Hours –ge 6) { #warning } else { #good } For the “stop light” effect, we will use the Write-Host command and adjust the foreground and background colors. For example, for critical we will use a background of red and foreground of white. Write-Host “CRITICAL:” –Backgroundcolor Red –ForegroundColor White The resulting output can be seen below. All of the other functions in this script are called and processed in much the same way, except for Get-AppLogEvents. Request #7 required me to scan the SQL Errorlog for any errors. While reading the contents of an Errorlog is simple, there is no real easy or efficient way to scan for specific errors or keywords. However, every time SQL Server writes an event to the Errorlog, it also writes the same event to the Windows Application Log. Knowing that, we can use Get-WinEvent to look for events that are classified as errors. One of the advantages to using Get-WinEvent is that we can use the FilterHashtable to filter our results on the target server before returning the results back to our client. This greatly reduces the amount of time to return results, but it also reduces the amount of data sent across the network. $events = Get-WinEvent –ComputerName ‘MySqlServer’ –FilterHashtable @{LogName=’Application’;Level=2;StartTime=((Get-Date).AddDays(-1));ProviderName=’MSSQL$Instance’} –ErrorAction SilentlyContinue In the example above, the FilterHashtable is used to pass four filters. The first gets the events from the Application log, the second only returns errors (Level=2), the third returns events that occurred within the past 24 hours, and the fourth filters on the source ‘MSSQL$Instance’. If you used Event Viewer, you could filter the same information by selecting these options in the picture below. This function works only if your SQL Server does NOT use the “-n” startup option. Using that option prevents SQL Server from writing events to the Windows Application Log. And there you have it; a simple PowerShell script to capture all that information from your environment. The complete script can be downloaded from GitHub. https://github.com/PatrickKeisler/SQLMorningHealthChecks Additional references: https://docs.microsoft.com/en-us/sql/relational-databases/server-management-objects-smo/sql-server-management-objects-smo-programming-guide https://docs.microsoft.com/en-us/sql/database-engine/configure-windows/database-engine-service-startup-options

Exploring Big Data: Course 9 – Developing Big Data Solutions with Azure Machine Learning

Samuel Lester - MSFT — Tue, 19 Dec 2017 12:09:37 GMT

(This is course #9 of my review of the Microsoft Professional Program in Big Data)

Course #9 of 10 – Developing Big Data Solutions with Azure Machine Learning

Overview: The “Developing Big Data Solutions with Azure Machine Learning” course is comprised of four sections of videos and labs, each with a set of 3 quiz questions, followed by a 15-question final exam. The labs in each section build upon the work done in the previous labs. The first section introduces Azure Machine Learning Studio, followed by the next three sections where you begin to use Azure ML Studio to create your first predictive experiments. Time Spent / Level of Effort: This course took me considerably more time than the previous couple courses due to the length of the labs and my high level of interest in this subject matter. If your goal were to complete this course as quickly as possible, you could easily skip all the labs and answer the quizzes/final exam questions by only watching the videos since there are no direct questions based on the lab activity or output. As I do with each edX course, I opened the homework quiz and the final exam in new windows to be able to watch for the content as it appeared in the videos. This allowed me to complete the course with 96% before beginning any of the labs. In total, I spent around 10-12 hours on this course. Course Highlight: When I completed the Data Science MPP in 2016, my favorite part of the program was the capstone project, where I spent a huge amount of time working in Azure ML Studio. Learning the intricacies of various predictive models, modifying test data, and tweaking model configurations made the final project extremely rewarding and kept me motivated to improve my model long after I obtained the required passing score. This course reminded me how much I enjoy building predictive models in Azure ML Studio. One additional highlight was when we published a predictive model as a web service, then called this web service directly within Excel on our existing data. Seeing this in action and understanding how easy it is to do is really a powerful exercise. This is a great example of the power of combining Excel with Azure ML and predictive analytics. Suggestions: The videos alone were sufficient to easily pass the course with the needed 70%. I watched them all at double speed with the quizzes and final exam open in other tabs to watch for the test content. Since the Data Science MPP capstone was spent almost entirely in Azure ML Studio, I’m hoping that the Big Data MPP will also do the same. If so, spending a lot of time carefully completing the labs and getting familiar with Azure ML Studio will be great preparation for the final capstone project. If you have taken this course in the past or are going through it now, please leave a comment and share your experience. Thanks, Sam Lester (MSFT)