A Story of BI, BIG Data and SQL Server in Canada https://docs.microsoft.com/archive/blogs/cansql/feed.xml We focus on BI, Big Data and SQL Server, providing the latest news from Microsoft and the industry, sharing upcoming events, customer stories and SQL Server product information. Find out why SQL and not Oracle or SAP. Thu, 11 Apr 2019 15:51:53 GMT en-US hourly 1 Are you ready for SQL Server 2008 and 2008R2 End of support? https://docs.microsoft.com/archive/blogs/cansql/sql-server-2008-and-2008r2-end-of-support Mon, 01 Apr 2019 15:34:45 GMT https://blogs.technet.microsoft.com/cansql/?p=6545 ]]> Operationalize Databricks Workloads with Azure Data Factory https://docs.microsoft.com/archive/blogs/cansql/operationalize-databricks-with-azure-datafactory Fri, 22 Mar 2019 02:40:57 GMT https://blogs.technet.microsoft.com/cansql/?p=6335 About the authors: Faraz Rasheed has been in the Advanced Analytics & AI space for over a dozen years and has led many innovative and bleeding edge initiatives with his customers. Alp Kaya focuses on the Azure Data Platform and works closely with customers to solution and ultimately onboard them on to Azure. Please reach out to us if you have any questions about the article. You can reach Faraz & Alp at: Hope you enjoy the article.

Overview

We are working with several customers who have adopted Azure Databricks as a platform for their Analytics, Data Engineering or Machine Learning requirements.  Azure Databricks is a great tool for these use cases and we are seeing accelerated adoption in the field.  This article is about operationalizing or productionizing Azure Databricks workloads with Azure Data Factory.  The Databricks team has done a superb job of enhancing open source Spark but has left the scheduling & orchestration type activities to a best of breed technology such as Azure Data Factory. So, in this blog, we will look at integrating Azure Data Factory with Azure Databricks and other Azure Data Services to achieve a production deployment. In this article, we will achieve the following objectives:
  • Step by steps instructions on how to operationalize a Databricks notebook (we will use the ML model that predicts survival of Titanic but this operationalization can be applied for other use cases such as Analytics or Data Engineering)
  • Scheduling the Databricks notebook to run on a schedule & orchestrate data movement between Azure Storage & Azure SQL database. Specifically in one Databricks notebook that:
    • reads data from Azure SQL DB & writes results back to Azure Storage
    • and then Azure Data Factory reads results from previous step (i.e. Azure Storage) and then writes results Azure SQL Database
 

 Architecture

Before getting started, let’s do a quick overview of the customer architecture and scenario along with relevant requirements.

Customer Scenario Overview

Customer has an on premise application where application is constantly making inserts and changes to an on premise SQL Server database.  The requirement is to migrate this data into Azure, apply Machine Learning on it with Databricks, and then write the results/outputs to an Azure SQL Database.  One of the reasons of moving the data from on premise to Azure is to leverage the Azure Data Services such as Databricks and Azure SQL but also to not interfere with the functioning of the SQL Server OLTP on premise application.  The above is a batch process expected to run nightly. At the end of the process, the outputs will be visualized on PowerBI.   Architecture Components Overview
  • Azure Databricks: Python development environment for training and scheduled scoring of ML models
  • Azure Data Factory: is a browser based tool that is an orchestration platform for building and scheduling Azure Databricks Notebooks (in our particular case - scoring pipeline)
  • Azure Storage: File storage services to hold model training data and model predictions before being pushed to Azure SQL DB
  • Azure SQL DB: Database platform used to store data for model predictions and to be the serving layer for PowerBI.
  • PowerBI will connect to Azure SQL DB in order visualize the ML models and other KPIs
Let’s see how we can in 5 steps productionize the above architecture with Azure Data Factory

Step 1 Create Azure Data Factory and Azure Databricks

From the Azure Portal, create a new Azure Data Factory Resource and Azure Databricks:

Step 2 Create a Databricks notebook

Our Databricks notebook will be scheduled to run on nightly basis and loads data from Azure SQL DB, creates new predictions by a pre-trained machine learning model and stores the results back to Azure storage account as a csv file.  A sample from the Databricks notebook (shown below) demonstrates the Azure SQL connectivity from the Azure Databricks Notebook via a jdbc connection.  Please note – the full notebook can be found in the Git repository here.

Step 3 Create Data Factory Pipeline

Azure Data Factory orchestrates production workflows, schedule and provide workflow management and monitoring services. In our case, we use Azure Data Factory to automate our production job running our scoring Databricks notebook to generate new predictions and loading these to our SQL DB table.  
  1. After setting up Azure Data Factory, create a pipeline and add Azure Databricks Notebook object and a Copy Data object to the pane as shown below
  1. Configure the Databricks Notebook object to connect with your Azure Databricks workspace and select the target notebook to run on schedule

Step 4 Configure the Copy Data Object

Copy Activity is also very straight forward. We pick our blob storage dataset as source, our SQL DB dataset as sink and define the field/column mapping for the two sources. Please note that we defined a pre-copy script in our SQL DB sink to delete the existing records in our target table. Since we already defined schema is both of our datasets, schema mapping in copy activity is also very straight forward. Such schema mapping becomes very handy when your source and sink datasets do not have matching or same number of columns.
  1. Configure the Source file with your azure blob storage
  1. Configure the Sink (or the destination). In this case it will be Azure SQL database
 
  1. Configure the Mapping tab in order to map source columns to destination columns as shown below:
 

Step 5 Publishing and Scheduling the Pipeline & Monitoring

With Azure Data Factory, we don’t ‘save’ our work but rather ‘publish’ our pipeline and validate its completeness with the ‘Validate’ link 
  1. Publish the Pipeline
2. Pipelines are scheduled by creating a ‘Trigger’ and setting up scheduling properties 3. Monitor the Pipeline One of the more powerful features of Azure Data Factory is its monitoring and management capabilities. It lets you monitor pipeline runs, their status (running, succeeded and failed) and to even drill down into the status of individual activities within pipeline. In case of an activity failure, we can drill down and in case of our Databricks notebook activity, we can even open the exact instance of Databricks notebook with cell status and error messages right within the notebook as shown below:  

Lessons Learned:

  1. We write our predictions first to Azure storage as a csv file which is then loaded to the Azure SQL DB database using Azure data factory. This helps ensure that issues like connection failures, incomplete writes and efficient data load is handled by a mature orchestration tool like Data Factory. This decoupling also adds other benefits like writing to multiple destinations and triggering another job not discussed in this blog
  2. Azure Databricks provides a built-in key-vault service called secret scopes to manage sensitive information like username and passwords without exposing these to code. Checkout our github page for details on how to set these up
  3. For SQL connectivity, we found JayDeBeApi works well with Databricks. The underlying VM running Databricks code uses UNIX, hence the ODBC connectors do not really work well.
  4. While Azure machine learning (see full contents here) provides a streamlined way of registering, packaging and deploying models, the architecture used in this blog with Azure data factory as an orchestration tool works very well for batch deployment scenario and is consistent with non-machine learning use cases.
 ]]> BYOL SQL & Windows Server license to the Cloud: Azure Hybrid Benefit (AHB) vs License Mobility. What is the difference? https://docs.microsoft.com/archive/blogs/cansql/byol-microsoft-server-license-to-the-cloud-azure-hybrid-benefit-vs-license-mobility-what-is-the-difference Mon, 18 Mar 2019 19:10:37 GMT https://blogs.technet.microsoft.com/cansql/?p=6155 When an organization deploys a solution built on the Microsoft platform to the Cloud, there are three options in regards to licensing the required Microsoft Server products:
  • Option One: Organizations can purchase cloud services that include the required server products (e.g. a fully loaded Windows Server virtual machine).
  • Option Two (BYOL): Organizations can purchase prepaid SQL/Windows Server subscriptions (1 or 3-year terms available), currently offered through Microsoft’s CSP program. 
  • Option Three (BYOL): Customers with active Software Assurance can bring their own SQL/Windows Server licenses to the Cloud. 
Options 2 and 3 are what we call Bring Your Own License scenario (BYOL), and they provide better TCO in most of the cases. For example, by bringing SQL Server, Windows Server licenses to Azure organizations can realize significant savings in Azure. The table below compares BYOL scenario vs buying SQL Server Enterprise license as a part of Pay As You Go Azure Virtual Machine subscription (please find the detailed view here): SQLblog1 The rules of BYOL are different for Microsoft Cloud (Azure) and other Clouds. The BYOL programs for Azure and other cloud providers also have different names. For other cloud providers program name is License Mobility because it actually allows moving licenses between on premises datacenter and the cloud provider. For Azure the BYOL program name is Azure Hybrid Benefit (AHB), and it is an extension of License Mobility offer. Key requirement to leverage AHB or License mobility offers is an active Software Assurance for the server product or an active server product subscription. The tables below summarize the key difference between AHB and Licenses Mobility for SQL Server and Windows Server.

SQL Server

Cloud type: Microsoft Azure Other Clouds (GCP, AWS, other hosting providers*)
Offer name: Azure Hybrid Benefit License Mobility
IAAS scenario:

BYOL to a Virtual Machine (VM) in the Cloud

 1 SQL Server ENT/STD Core license provides the rights to run 1 vCore in the  Azure VM.

Each VM in the Cloud should have at least 4 SQL Server Core Licenses assigned.

For example, if you have only 2  vCore VM you still need to ssign 4 SQL Server Core licenses to this VM.

[The same as for Azure Hybrid Benefit] 1 SQL Server ENT/STD Core license provides the rights to run 1 vCore in the Cloud VM.

Each VM in the Cloud should have at least 4 SQL Server Core Licenses assigned. 

For example, if you have only 2 vCore VM you still need to assign 4 SQL Server Core licenses to this VM. 

PAAS scenario:

BYOL to a SQL Server Managed Service (shared infrastructure)

 Yes, organization can bring SQL Server licenses with active SA to Azure SQL Managed Instance.If you have Enterprise Edition per core licenses, you can get 4 vCores in the Azure SQL Managed Instance General-Purpose Tier for every SQL license core you own on-premises. This is a unique benefit available only on Azure. If you have Standard Edition per core licenses with Software Assurance, you can get one vCore in the general-purpose tier for every SQL Server license core you own on-premises. No, BYOL SQL Server license scenario is not available for Managed services such as AWS RDS and others.
Can we use our SQL Server license on-premises AND in the cloud simultaneously? No, your SQL Server licenses must be used either on-premises OR in Azure. 

However, you have a 180-day grace period during which you can use licenses both on-premises and in the cloud to facilitate migration.

 No, your SQL Server licenses must be used either on-premises OR in the cloud.

 

Windows Server 

Cloud type: Microsoft Azure Other Clouds (GCP, AWS, other hosting providers *)
Offer name: Azure Hybrid Benefit   License Mobility
IAAS scenario: BYOL to a Virtual Machine in the Cloud Azure Hybrid benefit is available for Windows Server.

For virtual machines running 24x7 this option provides significant savings compared with buying SQL Server subscription along with VM.

 No License Mobility (BYOL) for Windows Server**. The Windows Server license  subscription can be acquired along with VM from the cloud provider.
Can our Windows Server license be used on premises AND in the cloud simultaneously? The Azure Hybrid Benefit for Windows Server Standard Edition licenses can only be used once either on-premises OR in Azure.

Windows Server Datacenter Edition benefits allow for simultaneous usage both on-premises AND in Azure.

Each 2-processor license or each set of 16-core licenses are entitled to two instances of up to 8 cores, or one instance of up to 16 cores.

 No License Mobility (BYOL).
  The differences between Azure Hybrid benefits and License Mobility listed above are one of the reasons why Azure is the most cost-effective cloud for Windows Server and SQL Server. clip_image005[7] If you are interested to learn more about SQL Server licensing please watch this webinar: Demystifying SQL Server 2017 and How to Buy.   I want to say Big Thank you to Jai Dhir, Microsoft Licensing Manager for reviewing the blogpost and providing valuable feedback. Please let me know if you have any comments, questions or feedback by sending email to CanadaDataPlatform@microsoft.com  _______________________ * please find the list of Authorized Mobility Partners here ** Windows Server is not eligible for License Mobility through Software Assurance, please find the details here This blogpost is for information purposes only, for the official Microsoft documentation on Azure Hybrid Benefit and License Mobility please check these links:      ]]> Use MS PowerPoint to record a video of your presentation https://docs.microsoft.com/archive/blogs/cansql/use-ms-powerpoint-to-record-a-video-of-your-presentation Wed, 03 Oct 2018 16:04:02 GMT https://blogs.technet.microsoft.com/cansql/?p=5975

Did you know that you can use PowerPoint to record a video of your presentations?

image

Presenter view (you can turn on/off your video and use markers):

image

Love this feature: You can redo the recording for specific slides when you are done!

image


Here is a step-by-step guide (thanks to Patrick Vuong for building it):

  1. Have your slides finished in PowerPoint
  2. To begin the recording: Go to File Export Create a Video.
  3. Select resolution required

clip_image002

  1. Hit the Drop down and select Record Timings and Narrations which will take you to a presenter view:
  • Check the Mic on the bottom right, click mute and then unmute to make sure it is working
  • Press “R” on your keyboard to start recording
  • Use your arrow keys to go to the next slide
  • Press “S” on your Keyboard to finish the recording

image

  1. Check after your recording that there is a Microphone at the bottom right of the corner (not visible if you recorded your video)

clip_image006

  1. Afterwards go back to Export Create a video, then make sure it is “ Use Recorded Timings and Narrations”.  Select resolution a create a video

clip_image008



The detailed instructions are here:

https://support.office.com/en-us/article/turn-your-presentation-into-a-video-c140551f-cb37-4818-b5d4-3e30815c3e83#OfficeVersion=Office365

https://support.office.com/en-us/article/record-a-slide-show-with-narration-and-slide-timings-0b9502c6-5f6c-40ae-b1e7-e47d8741161c

]]> Are you ready for SQL Server 2008/R2 End of Support? https://docs.microsoft.com/archive/blogs/cansql/are-you-ready-for-sql-server-2008-r2-end-of-support Mon, 01 Oct 2018 20:25:24 GMT https://blogs.technet.microsoft.com/cansql/?p=5795 image

SQL Server 2008 and SQL Server 2008R2 end of support is coming in just 9 months. Are you ready?

Join From Data to AI webinar series to learn how you can get 3 extra years of free security updates and migrate your SQL Server 2008/R2 workloads to Azure:


Register here: www.aka.ms/FromDataToAI

]]> From Data to AI webinar series https://docs.microsoft.com/archive/blogs/cansql/from-data-to-ai-webinar-series Fri, 14 Sep 2018 17:26:30 GMT https://blogs.technet.microsoft.com/cansql/?p=5725 ms_2125_datatoai_regbanner_V7_ON24

Get Your Data Ready For AI

AI is useless without well-organized data, that is what Data Governance is about. Did you know? Most data scientists spend only 20 percent of their time on actual data analysis and 80 percent of their time finding, cleaning, and reorganizing huge amounts of data, which is an inefficient data strategy.

Is your data ready for AI and Analytics? Are you satisfied with your data security, availability and your applications performance? Is your organization ready for upcoming SQL Server 2008/2008R2 End of Support? Join our webinar series to learn how you can improve Security, Data Governance and Information Quality with SQL server 2017 and Azure SQL services.

Transform Your Business with AI and IoT

Artificial intelligence (AI) makes it possible for machines to learn from experience and perform human-like tasks. Using AI and Data Science, computers can be trained to accomplish specific tasks by processing large amounts of data and recognizing patterns in the data.

Additionally, implementation of Internet of Things capabilities within your organization allows you to leverage AI to optimize your physical operations and turn device and sensor data into true insight. Join these webinars to learn how transforming your critical business processes & applications with AI & IoT to innovate & evolve ahead of your competitors.

How to Buy?

You are ready to start a project and now you want to learn how you can acquire Microsoft product and servicers to identify most cost-effective option. Please join this webinar to learn about Microsoft products and services pricing and how to buy


Are you thinking of deploying a Modern BI, Predictive Analytics, AI or Big Data solution? Is it time to upgrade from SQL Server 2005/2008, move to the Cloud or consolidate your data infrastructure to improve the performance, optimize the cost and achieve higher flexibility? Interested in migrating from Oracle, Sybase or DB2 to save money and improve security?

If you answered yes to at least one of these questions, then you might be interested in a complimentary Workshop, Assessment or Proof of Concept. Learn more

image

*No purchase necessary. Contest open to residents of Canada excluding Quebec and Public Sector employees. There are 3 prizes available per webcast (total of 36 prizes) to be won consisting of Microsoft Branded Laptop Bag (est. retail value $70). Odds depend on number of entries. Selected participant is required to answer correctly a skill-testing question. Entry deadline <11:59PM> ET on <December 20, 2018>. Limit 1 entry per person/webinar and limit 1 prize per person. For complete details, see Official Contest Rules available www.aka.ms/2018rules.

]]> Microsoft AI resources https://docs.microsoft.com/archive/blogs/cansql/microsoft-ai-resources Mon, 25 Jun 2018 04:25:41 GMT https://blogs.technet.microsoft.com/cansql/?p=5675

Founded this great list of Microsoft AI resources in the Introduction to Artificial Intelligence (AI) course on edx.org.

image

Cognitive Services

For an overview of the Cognitive Services APIs, take a look at Getting Started with Microsoft Cognitive Services on Channel 9. Then you can dive into the documentation for individual Cognitive Services APIs, which often includes tutorials to help you get started (note that this is not an exhaustive list!):


Building Bots

For in-depth training on building bots with the Bot Framework, there's a course on Microsoft Virtual Academy:

You may also find the following documentation useful:


AI Engineer Resources

If you're an aspiring AI engineer, and you want to continue learning about machine learning, deep learning, and applied AI solutions, you can take the following courses to develop the skills necessary to succeed.

]]> Replace your SAP BW by leveraging Simplement and the Microsoft Azure Data Stack https://docs.microsoft.com/archive/blogs/cansql/replace-your-sap-bw-by-leveraging-simplement-and-the-microsoft-azure-data-stack Wed, 02 May 2018 15:11:19 GMT https://blogs.technet.microsoft.com/cansql/?p=5635 If you’d like to discuss the article or any Azure Data related services please reach out: https://www.linkedin.com/in/kayaalp/

Objective of the Article?

SAP ERP is one of the most widely used business applications in Large Enterprise (LE). It is great piece of software for business processes and so wide spread in the market that 2/3 of the world’s GDP runs through an SAP system. However, one of things it is not known for is its ability to allow users access to the data and enabling analytics on the ERP system. SAP – recognizing this as a short coming, developed an application called SAP BW (Business Warehouse) which is a prebuilt proprietary Data Warehouse application. The licensing for the BW application is free so long as you have users licensed on SAP ECC. SAP BW solution provided SAP customers standard extractors and prepackaged analytics/reports that would read/extract directly from you ECC system (+ other data sources as well) and load SAP BW analytics structures. The prepackaged and out of the box analytics reports would fulfill common reporting requirements such as Sales, A/R, A/P, Financials, etc…However, the SAP BW system is based on the same proprietary SAP ERP platform Netweaver and ABAP. The overall market and customer perception SAP BW application has been luke warm at best – there are customers who absolutely love BW and others who refuse to leverage it and would rather spend the energy and money and build their own. SAP recognizing the marketplace perception and attempted to modernize SAP BW developed with an in memory DBMS called SAP Hana. SAP Hana allowed SAP BW (+ SAP ECC in subsequent releases) application to run on top of it. In the latest reincarnation, SAP has announced a new transactional product (S/4) but has not been widely adopted yet. The predominate use case for SAP Hana has been to move SAP BW workloads onto SAP Hana. The SAP Hana solution enables the SAP ERP or SAP BW in memory and hence was able to accelerate the data access to business users. In addition, with SAP Hana, a very SQL like interface for the SAP system and this really simplified access so long as you have the Hana Enterprise (i.e. non-runtime). Some customers chose to deploy SAP Hana as a side car and replicated data from SAP ERP straight into it via SLT and other tools. This deployment model constituted a full use (non-runtime license). And although SAP Hana is a great platform, it is a very expensive software license and requires a very expensive hardware appliance. It’s not unusual for customers to spend over 7 figures on licensing and hardware for a couple of TB of Hana.

Overview and the Challenge?

Most recently, I have had the opportunity to work with a customer who has a multi-terabyte ECC and an SAP Hana side car deployment. This customer’s preference was to build their own data warehouse (instead of using SAP BW) via SAP Hana side car. This customer mightily struggled with getting access to Financial Reporting out of their SAP ECC system and by struggled, I mean:
  • It took more than 6 months to build a baseline Financial Income Statement
  • Several years to complete the finalized Financial Income Statement (albeit the final Financial had over 200 measures that included several this period vs prior period vs same period prior year and all of the % increases/decreases of each – which are very computationally taxing)
  • Cost several million dollars to generate and maintain this one single Financial Statement report (for a variety reasons from data integration into SAP Hana via SLT & Data Services, Calculation Views on top of Calculation Views, and the general complexity of the solution)
This customer was wondering if there was a better, easier and more cost effective way to do this both from a licensing, hardware, and resourcing perspective. Working closely with this customer, we engaged in a 6 week project and leveraged an ISV solution called Simplement which produced some incredible results.

What does Simplement Do?

Simplement is a metadata driven solution that comprehends customer's SAP ERP including all custom Z objects and automatically transforms the SAP data and exposes the underlying tables via a common SQL interface. The Simplement solution first takes the underlying SAP tables, de-clusters SAP cluster tables, de-pools SAP pooled tables, and translates German language SAP columns/tables into English and makes them available as regular tables in Microsoft SQL DBMS. The Simplement Data Liberator (core component of the Simplement solution) also automatically manages SAP hierarchies and allows you to automatically inherit your existing SAP security model to the Liberated data. So, for customers who have large user communities where security has already been or for customers who have well defined hierarchies and want to continue leveraging them, these are all cascaded to the SQL world without any re-work. Once this is done, all of the data in SAP can now easily be queried via a very common non-proprietary SQL language on the Microsoft SQL DBMS. Simplement then provides out of the box multi-dimensional views (think OLAP) into the data in the form of in memory Microsoft Analysis Services Tabular models. Microsoft Analysis Services Tabular is a multidimensional in memory analytical structures and can be thought of as a competitor to SAP Hana. Furthermore, the Simplement solution provides 9 multi-dimensional Analysis Services Tabular LOB models (known as “Smart Starts” covering areas such as S & D, GL, A/R, A/P, COPA, and others. Finally, to get you going with the Analysis Services Tabular models, the Simplement solution provides a prebaked PowerBI reports for each of the different smart start areas. You can use PowerBI to build management reports, and your company analysts can also connect Excel directly to the Tabular Smart Start model for deep analysis. The Data Liberator can also feed the transformed SAP data to Azure Data Warehouse, Machine Learning, Cognitive Services, etc.

Key Components and Highlights of The Solution

clip_image002[3]

Key Highlights of the Solution

1. Out of the box Microsoft SQL Server that contains the SAP data in a SQL table format. With SQL, SAP documents can be searched from a variety of methods. For instance, PowerBI or Excel can be connected and be used as front end to search for any SAP document or analysis can be done straight on the SAP data in SQL. In addition, with Microsoft SQL 2017, both Python and R can be leveraged to execute statistical and data science packages without having to export to a separate package. Lastly, the Azure Search as a Service functionality can be used so that “search” can be applied on SAP data in the SQL DBMS. clip_image004[3] For instance, imagine navigating your customer data or part masters data in a ‘google’ style search. The image above is an example of Azure Search as a Service with the following features turned on providing a rich UI experience on the SAP data: § Full text search is a primary use case for most search-based apps. Queries can be formulated using a supported syntax. § Search suggestions can be enabled for type-ahead queries in a search bar. Actual documents in your index are suggested as users enter partial search input. § Faceted navigation is enabled through a single query parameter. Azure Search returns a faceted navigation structure you can use as the code behind a categories list, for self-directed filtering (for example, to filter catalog items by price-range or brand). 2. Out of the box, the Simplement solution provides several in memory Microsoft AS Tabular models such as S&D, General Ledger, COPA, A/R, A/P, Inventory, Asset History, etc… These models provide transactional facts, conformed dimensions, and key measures for your SAP data. In addition, using the DAX language in AS Tabular, additional KPI and measures can easily be created. One of the powerful features of the DAX language is the ability to easily create key measures like this period vs prior period or this period vs same period last year. Often, this type of requirement takes long periods of time to develop in other tools, but this is an inherent feature of the AS Tabular solution. clip_image006[3] 3. Simplement provides out of the box PowerBI reports for modules like S&D, General Ledger, COPA, A/R, A/P, Inventory, Asset History, etc... So, various LOBs can get access to the SAP data and do in depth analysis (drill up/down and across) without having to do any data integration or custom development. Crucial SAP data elements are exposed as prebuilt PowerBI dashboards and reports. Lastly, this content can further be customized with additional calculations and different visualizations. clip_image008[3] 4. Once the data is in the Microsoft Cloud (i.e. Azure), the possibilities become endless. For instance, IoT Hub can be leveraged to pull in terabytes/petabytes of data and marry that back with Asset/maintenance data from SAP. Or, Azure ML can be leveraged to apply Machine Learning on the data with endless possibilities like predictive maintenance, financial forecasting, and others. The diagram below depicts some of the possibilities once you are on Azure: clip_image010[3]

Project Results

1. Within a week a fully functional AS Tabular GL Model on commodity hardware on Azure 2. A baseline Financial Income statement within a week after accessing SAP ECC with over a 100 key relevant measures out of the box 3. Leveraging the Azure Data Stack and applying advanced analytics via R and forecasting terabytes worth of Financials within a few minutes

Final Thoughts

Simplement is a great way to provide analytics and easy ad-hoc requests to SAP ERP systems. Simplement works completely transparent of the underlying databases - even if the SAP ERP system is running off Hana. Finally, the Simplement solution provides the out of the box SQL DBMS, Analysis Services Tabular (in-memory) running on commodity hardware, and PowerBI content to get you started. This also unlocks the ability to take a traditional SAP customer and move them to Microsoft Azure – opening with it all that Azure can do. For instance, applying Azure Search as a Service to SAP documents, applying Big Data or IoT, and Machine Learning to SAP ERP data along with elastic scale up/down of the cloud.]]> Missed Microsoft’s free workshops - “From RDBMS to NoSQL: A look at MySQL, PostgreSQL and Cosmos DB on Azure”? https://docs.microsoft.com/archive/blogs/cansql/missed-microsofts-free-workshops-from-rdbms-to-nosql-a-look-at-mysql-postgres-and-cosmos-db-on-azure Mon, 15 Jan 2018 18:04:34 GMT https://blogs.technet.microsoft.com/cansql/?p=5385

The week of Jan 22nd, we hosted a workshop tour in five cities across Canada focused on database options on Azure, not including SQL. As you may notice from the event title, the workshop dug into providing an overview of use cases and scenarios, along with hands-on experience of working with the community editions of MySQL and PostgreSQL databases, fully managed by Azure. Additionally, for NoSQL fans, the workshop dove into leveraging existing investments you have made into OSS frameworks and languages with Azure Cosmos DB – a globally-distributed, multi-model database service that is schema-less and generally classified as a NoSQL database.

In this blog post, you’ll find links to self-paced labs, videos, documentation, trial options and a way to obtain assistance from Microsoft partners for complimentary assessment or proof of concept.

For either continued learning after attending the workshop, or because you missed the workshop, I recommend the following hands-on labs that you can do without an Azure subscription:

Prefer short videos for learning?

Check out these quick links:

Looking for some light night time reading?

Check out the following documentation including tutorials, quick starts for multi-model APIs, how-to guides and lots of other great information:

· Azure Cosmos DB documentation

· Azure Database for MySQL documentation

· Azure Database for PostgreSQL documentation

Trial by Fire?

And finally, if your preference is to roll-up your sleeves and just start building, start now by giving it a test drive on Azure:

For organizations looking for assistance from a Microsoft partner for their data and/or analytics projects, you may request a complimentary workshop, assessment of PoC at: https://aka/ms/data6offers

Thank you for your interest and we hope to continue to bring you more open source solutions on Azure!

Your Open Source on Azure Team at Microsoft Canada

CanadaOpenSource@microsoft.com

]]> Microsoft BOT and AI URL Whitelists https://docs.microsoft.com/archive/blogs/cansql/microsoft-bot-and-ai-url-whitelists Wed, 03 Jan 2018 19:43:49 GMT https://blogs.technet.microsoft.com/cansql/?p=5375  

 

When working with the Microsoft BOT Framework and different types of artificial intelligence algorithms, customers typically want to understand what they may need to whitelist in terms of URL access.

The following table is a list that I have compiled for my customers:

 

Resource URL to whitelist
Azure Portal portal.azure.com
LUIS

*.api.cognitive.microsoft.com

Deploying bot code *.scm.azurewebsites.net
General Azure *.azurewebsites.net

login.microsoft.com

BOT Framework

*.botframework.com

 

Note that this does not include any of the channels i.e Skype, Yammer etc. where the bot may live.

]]>