Gandhali's blog https://docs.microsoft.com/archive/blogs/gsamant/feed.xml Commit to your Code !! Wed, 17 Apr 2019 03:47:26 GMT en-US hourly 1 Create an End-to-end IOT Scenario with Azure Services https://docs.microsoft.com/archive/blogs/gsamant/create-an-end-to-end-iot-scenario-with-azure-services Fri, 24 Feb 2017 18:04:30 GMT https://blogs.msdn.microsoft.com/gsamant/?p=75
  • Things - Things can be anything that can send some kind of useful data. It can be a simple phone or a complex array of sensors sending the data.
  • Connectivity - These things need a way to communicate with each other and with internet and for that they need connectivity and protocols.
  • Data- The data can be a simple telemetry data, an image or even a video feed.
  • Analytics- The data that is gathered needs to be analyzed to derive insights from it. These analytics can be real time analytics such as sending an alert or taking an action or a long term analytics where you might use machine learning on historical data to detect a pattern or predict an outcome.
    • So how would a typical event processing would happen in an IOT Scenario. Lets a take a look. image1 Starting with producers, these are your devices or things which would be sending data. The data can either be directly sent to cloud if the devices are internet enabled. If the devices are legacy devices and not internet enabled, then some kind of gateways or aggregators can be used to collect the data from all such devices and then send it to cloud. Then you need an ingestion mechanism - a service or solution which is capable of ingesting data from any number of devices at a time and scale as per the need. Next, you would need a service which would process the incoming data, either to generate a real time action or to aggregate it and store it for further analysis. You would also need  data storage capabilities to store the huge amount of data that will be coming in and at the last you might want to have a way to visualize, search or query this data or to use it for your machine learning experiments. How Azure fits into the picture ? Microsoft Azure services give you a way to create your entire IOT solution with the use of many different components. Its not a one way solutions but rather a platform consisting of different services from which you can choose the services best fitted for your own scenario. Here is just a glimpse of the all the available services that can be used in your IOT Scenarios. Microsoft Services Let's build a scenario, shall we? We will create a simple IOT scenario, where we will gather data from different devices and ingest it in cloud. Once it is ingested, we will analyze the data to detect an anomaly and then raise an alert. We will use following Azure services in this scenario. Scenario Step 1 - Devices Azure IOT Hub has a way to interface with many different types of devices. This repository contains both IoT device SDKs and IoT service SDKs. Device SDKs enable you connect client devices to Azure IoT Hub. Service SDKs enable you to manage your IoT Hub service instance. https://github.com/Azure/azure-iot-sdks The SDKs are available for different programming languages such as C, Python, Node.js, Java, .Net and also can be used with a broad range of OS platforms and devices such as Linux, Windows and real-time operating systems. The other way to connect to IOT Hub is via IOT Hub Rest APIs if for some reason you can not use a SDK. In this scenario, we are going to simulate a device using SDK for .NET. This simulator will generate random values in a given range and send those values to Azure IOT Hub at a specific Interval. The Link to Code https://github.com/gsamant/IOTEndtoEndDemo Step 2- Create and setup Azure IOT Hub Azure IOT Hub helps you to establish bi-directional communication with billions of IOT devices and to set up individual identities and credentials for each of your connected devices. (You need an active Azure subscription to create Azure IOT Hub) Create an IOT hub using steps described here. (For this scenario, follow the steps only till the creation of IOT Hub.) Once your IOT Hub is created, the next step is to register a device, this can be done via Azure SDKs, Rest APIs or Azure CLI. You can also use the tools such as Device Explorer (Windows) or IOT Hub  Explorer. Once the device is registered you will have a device connection string which will have IOT Hub Name, Device ID and a Shared Access Key unique to your device. Once you have the connection string, use the values from the connection string to update the following lines in Program.cs file static string iotHubUri = "<IOT Hub URI>"; static string deviceId = "<Device ID>"; static string deviceKey = "<Device Shared access key>"; Step 3- Create and setup Azure Event Hub Azure Event hub is a managed service that can intake and process massive data streams from websites, apps and devices. This service can be used to directly used to ingest data from devices as well however unlike IOT hub, Event Hub can handle only device to cloud communications. In this scenario however, we are going to use Azure event hub as an output for the Azure stream analytics job. Create an Event hub using steps described here. And Note the connection string. Step 4 - Create a Stream Analytics job Azure Stream Analytics is  a fully managed real-time event-processing engine. Events can come from sensors, applications, devices, operational systems, websites, and a variety of other sources. Create a Stream Analytics job using steps described here. (For this scenario, follow the steps only till the creation of Stream analytics job). You will have an empty job as below Stream Analytics Once the Stream analytics job is created, Add the input, output and update the Query as shown below. [caption id="attachment_135" align="alignleft" width="300"] Input[/caption] [caption id="attachment_145" align="alignleft" width="300"]Output Output[/caption] [caption id="attachment_155" align="alignnone" width="1024"]Query Query[/caption] Start the Stream Analytics Job. Step 5 - Create an Azure function to process the Event hub queue Azure Function App is a serverless event based Compute service. The processing can be triggered via an event, a timer or a SaaS activity. In this scenario, we will create an Azure function which will be triggered when the event hub defined in Step 4 receives an event.
    1. Go to the Azure portal and sign-in with your Azure account.
    2. Search for Function App and Click Create. [caption id="attachment_165" align="alignnone" width="300"]Azure Function Creation Azure Function Creation[/caption]
    3. In your function app, click + New Function > EventHubTrigger - C# > Create. This creates a function with a default name that is run on the trigger from an Event Hub. It will also ask you to add the name of your Event Hub and the Event hub connection string. (For connection string click new, give a name to your connection and paste the connection string from Step 4)
    4. Copy the code from ProcessEvent.txt file from the GitHub link [caption id="attachment_175" align="alignnone" width="800"]Azure Function Code Azure Function Code[/caption]
    5. Leave the Client.BaseAddress URL blank for now. We will update this once we create the Logic App.
    Step 6 - Create an Azure Logic app to send an Alert Azure Logic Apps enable you to develop integration solutions with ease and let you automate and simplify business workflows across on-premises and the cloud. In this Scenario we will create a Logic App which will be triggered from an HTTP request and will send SMS using a Twilio Connector.
    1. On the Azure portal dashboard, select New.
    2. In the search bar, search for 'logic app', and then select Logic App. You can also select New, Web + Mobile, and select Logic App.
    3. Enter a name for your logic app, select a location, resource group, and select Create. If you select Pin to Dashboard the logic app will automatically open once deployed.
    4. After opening your logic app for the first time you can select from a template to start. For now click Blank Logic App to build this from scratch.
    5. The first item you need to create is the trigger. This is the event that will start your logic app. Search for Request in the trigger search box, and select it.
    6. In the Request Body JSON Schema field, paste following Schema. [caption id="attachment_185" align="alignnone" width="600"]JSON Schema JSON Schema[/caption]
    7. After you Click Save. The HTTP Post url will be generated. Copy this URL and paste it in Client.BaseAddress URL in Azure Function that is created previously. [caption id="attachment_195" align="alignnone" width="700"]Logic App -Request Logic App -Request[/caption]  
    8. Before going for the next step, we will need to create a Twilio Account. You can create a Free trial Account here. Before moving to next step, you need to have a verified Twilio Phone number that can send/receive SMS.
    9. Click on New Step and search for Twilio. Select Twilio-Send Test Message (SMS)
    10. Enter a connection Name for Twilio, Twilio Account ID and Twilio Access Token (You can get these two from your Twilio dashboard.) [caption id="attachment_205" align="alignnone" width="700"]Twilio Account Setup Twilio Account Setup[/caption]
    11. Once the connection is saved. Enter the to and from Phone numbers and Text for message. [caption id="attachment_215" align="alignnone" width="700"]Twilio SMS Setup Twilio SMS Setup[/caption]
    12. Save the Logic App changes.
    So far, we are done with the setup of our scenario. Lets test the scenario. Testing the scenario
    1. Run the SendEvent application. You will see events going to the IOT hub. [caption id="attachment_225" align="alignnone" width="600"]App Run App Run[/caption] Stop the application and in Program.cs file, uncomment the following lines. This will allow you to manually enter the Temperature value more that 60. Code snippet [caption id="attachment_235" align="alignnone" width="600"]App Run App Run[/caption]
    2. Check the SMS Alert on your phone. [caption id="attachment_265" align="alignnone" width="200"]SMS Alert SMS Alert[/caption]
    3. Also, check the Azure Function log. [caption id="attachment_245" align="alignnone" width="800"]Azure Function Log Azure Function Log[/caption]  
    Recap We created a simple scenario where we simulated a device sending telemetry information. Then we used IOT Hub to ingest the data from the device and Stream analytics job to process the data in real time. The stream analytics query, checked if the temperature is going above a threshold and when it did it sent that event to an Event hub. We then created an Azure function which will get triggered as soon as there is a new event on the Event hub and will send the Event data to a Logic App using HTTP Request Endpoint. On the Logic App trigger, we created a Twilio connector which will read the temperature from the payload and send the Alert.          ]]>         Ping and Tracert commands on Azure VM https://docs.microsoft.com/archive/blogs/gsamant/ping-and-tracert-commands-on-azure-vm Mon, 16 Feb 2015 05:56:00 GMT https://blogs.msdn.microsoft.com/gsamant/2015/02/16/ping-and-tracert-commands-on-azure-vm/ When data is sent over the Internet, it's sent in small blocks of data, called packets. Messages are divided into packets before they are sent, and each packet is then transmitted individually and can even follow different routes to its destination. Once all the packets forming a message arrive at the destination, they are recompiled into the original message.

    Sometimes when you are trying to send or receive data over the Internet you may experience timeouts, or a Web server may be down, which prevents you from accessing services and Web sites. There are two common programs that can be used to test your Internet connection and even help you diagnose congestion between your computer (your ISP) and the destination server you're trying to reach. The programs you can use are called Ping and Traceroute.

    The ping command is a very common method for troubleshooting the accessibility of devices. It uses a series of Internet Control Message Protocol (ICMP) Echo messages to determine Whether a remote host is active or inactive, the round-trip delay in communicating with the host and the packet loss.

    The traceroute command is used to discover the routes that packets actually take when traveling to their destination. The device sends out a sequence of User Datagram Protocol (UDP) datagrams to an invalid port address at the remote host.

    Now There might be a scenario, where you have to use these commands from Azure VM. ICMP support in Azure is blocked externally but you can allow ICMP on firewall rules in the VM and ping between VMs in the same cloud service or virtual network. If you wish to ping between Azure VMs and on-prem machines; then you can setup Point to Point IPsec tunnels or a Site to Site VPN, enable ICMP on both ends and ensure gateway-device firewall (or software VPN gateway) rules allow ICMP traffic (and have necessary routing table entries).

    On more way is to use TCP for the same. There are few tools available for that which come in handy.

    For Pinging –

    PSPing –You just need to download, unzip and use it (copy it to your VM for outbound pinging).

    Download - https://technet.microsoft.com/en-us/sysinternals/jj729731.aspx

    For Trace route -

    You can use a tool called NMAP. Again you need to download and install it. This also uses TCP port. What’s interesting is, it blocks some part of the route, maybe when it hits the datacenter.

    Download- https://nmap.org/download.html

     

     

    ]]>         Azure chat update #AskMSDevs https://docs.microsoft.com/archive/blogs/gsamant/azure-chat-update-askmsdevs Wed, 06 Aug 2014 05:02:00 GMT https://blogs.msdn.microsoft.com/gsamant/2014/08/06/azure-chat-update-askmsdevs/ What’s the first thing that you think about when you hear the word ‘Cloud’ or maybe the word ‘Tweet’?

    If you are a nature lover, you will think of some greenery,nice blue sky with white clouds and maybe chirping sound of birds. But if you are one of those tech geeks, you will probably imagine a big data center with lots of machines and maybe hashtags and At signs.

    Well for those who fall in the second category, @MSDevIndia team hosted a 2 hour chat about “Creating highly available and scalable Apps and APIs using Microsoft Azure”. We were joined by some enthusiastic developers who asked some really good questions but with a word limit set of 140 characters, not all questions could get a detailed answer. Below are some excerpts from the chat where we tried to elaborate on the answers.

    1.       What are the benefits of using Azure host my mobile app backend? Is there any place to learn about this?

    With Azure Mobile Services you can create a secure and highly scalable backend very quickly. And it supports all the platforms be it Android, iOS or Windows. It also gives you way to configure user authentication via Facebook, Twitter, Google or Microsoft. The notification hub gives you a way to push millions of notifications in a short time. And you can also write your own server side like using Node.js or C#.

    Some good references on Mobile services –

    https://azure.microsoft.com/en-us/documentation/services/mobile-services/

    https://channel9.msdn.com/Series/Windows-Azure-Mobile-Services

     2.       How feasible and easy to use Microsoft Azure mobile services as a back-end for game development (leaderboards and multiplayer)?

    Using mobile services for back end of games is very easy and quick. If you want an example check out this sample to see how quickly you can build it.

    https://code.msdn.microsoft.com/windowsapps/Adding-a-Leaderboard-to-1f9d216d

     3.       Is the Microsoft Azure credits of Rs. 12,100 that we get with a free trial valid for just a month or till the credits expire?

    This credit is valid for a month only. If you exceed your ₹12,100 free credit, your free trial account will be suspended. You can optionally upgrade your trial to be a Pay-As-You-Go Azure subscription at this point if you want to continue using and paying for services. If not, don’t worry – you won’t be billed anything.

    4.       What are the health monitoring and alerts that Microsoft Azure has to offer?

    For every VM, cloud service or storage account that you create you have a monitoring dashboard available in the portal. You can customize the monitoring by adding/deleting logging as well as configuring different metrics. Also, in the management portal, you have management services where you can create alerts for any of your apps and also view operational logs.

    5.       Can I as an iOS dev. make use of Azure?

    Yes. You can use Mobile Services, notification hubs and also Media services for your iOS Apps. https://azure.microsoft.com/en-us/documentation/services/mobile-services/https://azure.microsoft.com/en-us/documentation/articles/mobile-services-ios-get-started-push/

    6.       Is there a tutorial on how to quickly build and deploy apps to the cloud?

    You can create Apps as Websites or as cloud services. Here is a tutorial series about quickly creating and deploying Websites in Azure.

    https://channel9.msdn.com/Series/Windows-Azure-Web-Sites-Tutorials

    If you also want to check how to use Cloud Services. Check out this tutorial.

    https://channel9.msdn.com/Series/Windows-Azure-Cloud-Services-Tutorials

     7.       How does azure help in scaling my mobile app after deploying?

    After you deploy your mobile service, you can go to Azure management portal and check the auto scale options. You can configure the auto scaling depending on the number of API calls or you can even create scheduled scaling.

    8.       What are the risks involved in migrating my apps to the cloud?

    There are few things you need to consider when you move to cloud.

    1. Managing Security of your application
    2. Integration of on premise and on cloud system.
    3. Availability and performance of your applications.

    Microsoft Azure provides you multiple levels of security for your data. See this https://azure.microsoft.com/en-us/support/trust-center/ site for more information. Azure also allows you to create hybrid clouds using virtual networks. Azure gives you horizontal and vertical scaling options as well as load balancing and traffic routing capabilities to ensure the performance and the availability of the app When you plan your migration activities consider using following approach-

    1. Assess your Applications and Workloads.
    2. Build the Business Case
    3. Develop the Technical Approach
    4. Adopt a Flexible Integration Model
    5. Address Security and Privacy Requirements
    6. Manage the Migration

    9.       I have just started building apps & would like to know how to leverage Microsoft Azure at a novice level.

    For building mobile apps, you will need backend. You can check out mobile services. If you are building Web apps, Check out the tutorials mentioned above which show you how to quickly create websites and deploy them.

    10.    How much cost will I incur to shift an office of 20 employees to the cloud? We have an e-commerce business.

    This will depend upon what applications are being used in the office? Are they internal or external applications? A good starting point will be to make an inventory of your current IT requirements and also the costs associated with their maintenance and updates.And then use the calculator on www.windowsazure.com to check the costs on Azure.

    One point of caution will be its not always apple to apple comparison. In an on premises setup you always provision for extra load as scaling up can not be done quickly. Whereas when you move to cloud, you can actually use the infrastructure that you need and quickly scale up or down when needed.

    11.    I am using the #Eclipse IDE & would like to know if #Azure supports it or if it only supports Microsoft IDE like VS? Is there any specific #Azure plugins I can get, if so where do I get & how much will it cost me?

    Azure supports different platforms and framework. This link https://azure.microsoft.com/en-us/downloads/ will give you SDK downloads for all the different platforms that are supported. For eclipse plugin, here is a link.

    https://java.interoperabilitybridges.com/articles/windows-azure-plugin-for-eclipse-with-java

    https://msdn.microsoft.com/en-us/library/azure/hh694271.aspx

     12.   What are the risks involved in building websites using nodejs with Web matrix on Microsoft Azure?

    While there are no known risks as such a good place to start would be here to make sure you build your website correctly.

    https://azure.microsoft.com/en-us/documentation/articles/web-sites-nodejs-use-webmatrix/

    Thank you all once again for your valuable time and participation. In case you have further queries, please feel free to reach out to me @s_gandhali.

     

    ]]>