Girish's Blog - Are you developing ON the cloud? https://docs.microsoft.com/archive/blogs/girishp/feed.xml Cloud, Cloud, Cloud Sat, 08 Sep 2018 13:42:31 GMT en-US hourly 1 Using Azure CLI to open Kubernetes dashboard https://docs.microsoft.com/archive/blogs/girishp/using-azure-cli-to-open-kubernetes-dashboard Sat, 08 Sep 2018 13:41:00 GMT https://blogs.msdn.microsoft.com/girishp/?p=2005 Please Note: For the pics below please click on them to see the picture with clear resolution. This post tries to capture the additional Steps.
  1. First download the CLI as mentioned in the above pic.
  2. Login from the Azure CLI
  3. Second set your active/default subscription to the correct one.
  4. Then get the cluster credentials.
  5. Open the K8S dashboard.
The below screen captures these in sequence. Open an Admin Command Prompt to run these in sequence.

Download the Azure CLI, and install it and login interactively.

  Open the browser and enter the code to complete the login. Close the browser window and come back to the command prompt.

Setting your active/default subscription to the correct subscription, get credentials and launch tunnel to cluster.

Browse to cluster dashboard.   This is the complete sequence I had to do before I could get to the cluster dashboard.]]> Adding SQL IaaS extension into a disconnected Azure Stack https://docs.microsoft.com/archive/blogs/girishp/adding-sql-iaas-extension-into-a-disconnected-azure-stack Tue, 04 Sep 2018 22:23:50 GMT https://blogs.msdn.microsoft.com/girishp/?p=1995 Download Marketplace items in a disconnected scenario Unlike connected scenario, where you just click on the link "Add from Azure", and everything happens automagically,  in the case of disconnected deployments of Azure Stack it is not as easy and straightforward to populate the Azure Stack marketplace. In this blog we are specifically talking about populating the stack's marketplace for enabling your end users to create SQL Server IaaS VMs. Note that SQL Server requires an extension, before the SQL Server can be provisioned. We are specifically talking about how to get this extension in place, so that SQL Server can be deployed. In general the BYOL and Pay-As-You-Go images which you downloaded from public Azure marketplace and then uploaded into stack, would show up in the Azure Stack marketplace. However, the SQL IaaS should not(will not) show up in the marketplace, it is not deployable from the Add Extension UX directly. It can only be deployed from a template or when using one of the marketplace SQL VM offers. For the specific case of adding this into the Azure Stack marketplace, the offline syndication documentation does not have enough information to lead to a successful upload.  The below is the correct PowerShell for the current extension. Replace the URI for the storage blob. Add-AzsVmExtension -VmOsType Windows -ComputeRole IaaS  -Publisher Microsoft.SqlServer.Management -Type SqlIaaSAgent -Version  1.30.0.0 -SourceBlob "<customer’s blob location>" Ofcourse make sure to have the correct version of PowerShell and as well as to import the Compute Admin module of the powershell before firing the above powershell. Once this is done, you should be able to provision a SQL Server in an Azure Stack subscription.]]> Oh! No! The Azure EA portal converted my MSDN/VSTS subscription into EA subcription https://docs.microsoft.com/archive/blogs/girishp/oh-no-the-azure-ea-portal-converted-my-msdnvsts-subscription-into-ea-subcription Fri, 08 Jun 2018 02:26:00 GMT https://blogs.msdn.microsoft.com/girishp/?p=1965 I ran into a customer situation where when the customer went to the Azure Enterprise Agreement (EA) portal and made himself an account owner and then created subscriptions, his existing MSDN/VSTS based Azure subscription got converted into an Azure EA subscription. Of course this was not a great outcome.

Root Cause of this issue.

The reason this happened is because the account with which the customer had logged into the Azure EA portal was also the owner of the MSDN subscription. So, when you do this, it will give a prominent warning as below.   (Prior to this customer changed the “alternate account” of the MSDN/VSTS subscription to another account – but it seemed the change took time to be effective in the backend). Anyway, as mentioned before if you are logged in with the same account as the MSDN subscription account and made yourself an account owner, then all MSDN subscriptions that are tied to that account will be converted to an Azure EA subscriptions. Now in some customer environments this is perfectly OK and in some situations, customers might not want the automatic conversions and would want to maintain their MSDN subscriber benefits separate from Azure EA benefits. Both are perfectly valid, and we are discussing how to keep the separation. Users can use their AAD credentials or MSA to activate their Visual Studio subscription. They can also add an alternate identity if needed. They can activate the monthly Azure credit they get as a benefit with that VS subscription using either identity (but they only get the monthly credit once). When they do activate their monthly credit, they cannot also be an Account Owner on their EA. Account Owners are those people who are authorized to create Azure subscriptions tied to the EA. This EA role is unrelated to ownership of VSTS accounts and is also separate from the Owner role for an Azure subscription. EA customers who assign Visual Studio subscriptions to users using their AAD identities should avoid adding these same users as (Azure) Account Owners on their EA.

How to Prevent this: -

The best approach would instead be to have others (i.e., those who don’t have a Visual Studio subscription OR use an account that does not have a Visual Studio subscription tied to it) be added as Account Owners on the EA and for those accounts to create the Azure subscriptions needed by the development teams. Once the Azure subscription is created, the Account Owner will by default have the owner role on that Azure subscription and can add others with the owner role as well—so they could add the Visual Studio subscriber’s AAD identity as an owner on the Azure subscription and everything works end-to-end for the subscriber using their AAD identity without losing the monthly Azure credit.

Process to backtrack

Best approach if you already had caused the monthly credit benefit Azure subscription to be converted onto their EA (and thus to lose the monthly credit) is to do as follows. (Note the below requires creating a support ticket) 1)      Unassign the VS subscription from the user’s existing identity 2)      Reassign the VS subscription to a different identity for the user, an identity that is not an Account Owner on their EA. 3)      Then the user can activate their monthly Azure credit anew, using this different identity.    ]]> Disk partition in Azure VMs - using ARM Templates to specify disk configurations https://docs.microsoft.com/archive/blogs/girishp/disk-partition-in-azure-vms-using-arm-templates-to-specify-disk-configurations Thu, 17 May 2018 03:25:06 GMT https://blogs.msdn.microsoft.com/girishp/?p=1955 PowerShell DSC OR Azure DSC OR Azure platform Custom Script Extension (or Chef, Puppet or Ansible) to do this. Following are some useful links on this topic. As mentioned the script itself can be part of the template. Execution happens on the VM and not at the ARM layer. Some additional references:-
  1. Custom Script Extension Schema (Windows)
  2. Custom Script Extension Schema (Linux)
  3. Template based deployment of Script extensions (Windows)
  4. Template based deployment of Script extensions (Linux)
There is also another technique where the sequence of commands you needed to be executed can be put inside an inline variable as below and then refer to this variable as the content of the configuration that needs to be done on the VM coming up and logging in the first time. 
"<FirstLogonCommands> 
	<SynchronousCommand> 
		<Order>1</Order> 
		<Description>Create diskpart input file</Description> 
		<CommandLine> powershell.exe -Command Write-Output \"select disk 2 ' create partition primary ' format quick ' assign \" | Out-File C:\\diskpart.txt 
		</CommandLine> 
	</SynchronousCommand> 
	<SynchronousCommand> 
		<Order>2</Order> 
		<Description>Create formatted partition</Description> 
		<CommandLine>diskpart.exe /s C:\\diskpart.txt
		</CommandLine>
	</SynchronousCommand> 
</FirstLogonCommands>"
  In the above fragment we are specifying two commands to be executed in an order. The first one creates the file, whose contents are used/fed to the second command - in this case the disk partition utility - diskpart.exe. These commands are synchronous in nature as opposed to being async, so that order of execution is preserved.   Putting all of this together, the following is an example of using two inline variables (a) unattendAutoLogonXML and (b) unattendFirstRunXML which are then passed to the WindowsConfiguration to be executed on VM boot.       "variables": { "unattendAutoLogonXML":"[concat('<AutoLogon><Password><Value>',parameters('adminPassword'),'</Value></Password><Domain></Domain><Enabled>true</Enabled><LogonCount>1</LogonCount><Username>',parameters('adminUsername'),'</Username></AutoLogon>')]",     "unattendFirstRunXML":"<FirstLogonCommands><SynchronousCommand><CommandLine>powershell.exe -Command Write-Output \"select disk 0 ' select partition 1 ' extend\" | Out-File C:\\diskpart.txt</CommandLine><Description>Create diskpart input file</Description><Order>1</Order></SynchronousCommand><SynchronousCommand><CommandLine>diskpart.exe /s C:\\diskpart.txt</CommandLine><Description>Extend partition</Description><Order>2</Order></SynchronousCommand></FirstLogonCommands>", ... ... ...         "osProfile": {          "computerName": "[variables('vmName')]",           "adminUsername": "[parameters('adminUsername')]",           "adminPassword": "[parameters('adminPassword')]",           "windowsConfiguration": {              "additionalUnattendContent": [                {                     "passName": "oobeSystem",                     "componentName": "Microsoft-Windows-Shell-Setup",                     "settingName": "AutoLogon",                     "content": "[variables('unattendAutoLogonXML')]"                 },                 {                     "passName": "oobeSystem",                     "componentName": "Microsoft-Windows-Shell-Setup",                     "settingName": "FirstLogonCommands",                     "content": "[variables('unattendFirstRunXML')]"                 }             ]           }]]> Strategies to realizing Cost Savings in Azure https://docs.microsoft.com/archive/blogs/girishp/strategies-to-realizing-cost-savings-in-azure Thu, 22 Feb 2018 16:38:21 GMT https://blogs.msdn.microsoft.com/girishp/?p=1935 This is a long post. Please scroll down Table of Contents Introduction. 2 Choosing the appropriate Compute Option in Azure. 3 Planned (Pro-Active) approach to Compute Optimization. 3 Laying down the scaffolding/structure so VMs are provisioned with cost optimization guidelines. 3 Governance. 4 Naming standards. 5 Using Role Based Access Control (RBAC) 6 Using Azure Policies. 6 Review Resource Group (RG) Costs periodically to mine opportunities for cost reduction. 7 Making use of Tactical platform capabilities. 7 Azure Hybrid User Benefit 7 Use Reserved Instances (RI) 9 Shutdown. 10 Right Sizing. 11 Azure Low Priority VMs. 11 Scaling (Auto) 11 Performance optimization/performance POV. 12 Choose the right storage. 12 Choose the right type of VMs. 13 Choosing the appropriate platform construct 13 REACTIVE Approach - things to do post deployment 14 Azure Advisor. 14 Cloudyn type of tools. 14 Check if deployments can be converted for Reserved Instance(RI) 15 Check if deployments can be converted for AHUB. 1 Study the entities within the RG.. 15 Azure Compute VMs – Capability and Design Guidance. 15 VM Capability Guidance. 16 Azure VM Design Guidance. 22 Usage Analysis Tools. 23    

Introduction

  This document provides a framework/structured approach to consuming Azure Compute resources. At a high level this is divided into (a) Proactive approach and (b) a tactical/reactive approach to the topic at hand. It is worth noting that beyond the pro-active/reactive approaches described here, studying, and arriving at the appropriate application architecture with the platform native constructs in mind is equally if not more important in optimized usage of all resources (human, time, DevOps cycles, compute costs etc.,). Of course, this should be done much earlier in the life-cycle instead of merely lifting and shifting and then looking for opportunities to save costs. Another important dimension to Azure compute cost savings is taking an organizational view of azure services consumed, and pooling resource usage across applications. Classical examples of this are: (a)    using SQL Elastic pools if multiple applications use Azure SQL Services (b)    using fewer App Service Environments (ASE) but packing them to the maximum density with 100s of Azure Web Applications per ASE  

Choosing the appropriate Compute Option in Azure

With a bewildering choice of options, the challenge is to figure out the appropriate if not the most optimum compute choice within the Azure platform for the application/solution that you want to deploy. There is indeed a bewildering choice of compute options in Azure which are wide and varied and cater to a wide variety of application OR rather processing needs. For instance, what compute should one choose for a CPU intensive, or say a memory intensive application(s)?  – In these cases, the choices are easy. What should one choose for Web applications – again there are easy choices. But what should one choose say for a combination of requirements such as (a) highly decomposed and componentized application, (b) sudden bursts, (c) high velocity for the development team when pushing code, (d) where the underlying requirement is to have the appropriate compute (not the old style VM construct) for the task at hand, (e) support cloud native constructs, (f) enable smooth DevOPS, (g) fault tolerance and resilience? This is not an easy task and both Infrastructure architects and application architects should take the time to study and understand the platform capabilities and utilize the correct Azure platform constructs for the solution deployment. Taking the easy approach of deploying to VMs should be the last resort and avoided at best, to save time and money for the long run. The following link captures some of the well thought out decision criteria depending on the task or tasks the application or solution needs to accomplish. Criteria for choosing an Azure Compute Option

Planned (Pro-Active) approach to Compute Optimization

Laying down the scaffolding/structure so VMs are provisioned with cost optimization guidelines

These are things mature organizations do, so that end consumers of azure resources are within well-established guidelines (guard rails). The scaffolding is essentially a framework that is a combination of governance rules (enforce-able) and best practices. The graphic below captures the essential components that are used to build such a scaffolding and bear in mind the evolving nature of the platform, as newer constructs and capabilities get added.    

Governance

When moving to Azure, you must address the topic of governance early to ensure the successful use of the cloud within the enterprise. Unfortunately, the time and bureaucracy of creating a comprehensive governance system means some business groups go directly to vendors without involving enterprise IT. This approach can leave the enterprise open to vulnerabilities if the resources are not properly managed. The characteristics of the public cloud - agility, flexibility, and consumption-based pricing - are important to business groups that need to quickly meet the demands of customers (both internal and external). But, enterprise IT needs to ensure that data and systems are effectively protected. In real life, scaffolding is used to create the basis of the structure. The scaffold guides the general outline and provides anchor points for more permanent systems to be mounted. An enterprise scaffold is the same: a set of flexible controls and Azure capabilities that provide structure to the environment, and anchors for services built on the public cloud. It provides the builders (IT and business groups) a foundation to create and attach new services. The Azure enterprise scaffolding is based on practices we have gathered from many engagements with clients of various sizes. Those clients range from small organizations developing solutions in the cloud to Fortune 500 enterprises and independent software vendors who are migrating and developing solutions in the cloud. The enterprise scaffold is "purpose-built" to be flexible to support both traditional IT workloads and agile workloads; such as, developers creating software-as-a-service (SaaS) applications based on Azure capabilities. The enterprise scaffold is intended to be the foundation of each new subscription within Azure. It enables administrators to ensure workloads meet the minimum governance requirements of an organization without preventing business groups and developers from quickly meeting their own goals. Governance is crucial to the success of Azure. This article targets the technical implementation of an enterprise scaffold but only touches on the broader process and relationships between the components. Policy governance flows from the top down and is determined by what the business wants to achieve. Naturally, the creation of a governance model for Azure includes representatives from IT, but more importantly it should have strong representation from business group leaders, and security and risk management. In the end, an enterprise scaffold is about mitigating business risk to facilitate an organization's mission and objectives. The following image describes the components of the scaffold. The foundation relies on a solid plan for departments, accounts, and subscriptions. The pillars consist of Resource Manager policies and strong naming standards. The rest of the scaffold comes from core Azure capabilities and features that enable a secure and manageable environment.

Naming standards

The first pillar of the scaffold is naming standards. Well-designed naming standards enable you to identify resources in the portal, on a bill, and within scripts. Most likely, you already have naming standards for on-premises infrastructure. When adding Azure to your environment, you should extend those naming standards to your Azure resources. Naming standard facilitate more efficient management of the environment at all levels. For naming conventions: Review and adopt where possible the Patterns and Practices guidance. This guidance helps you decide on a meaningful naming standard. Use camel Casing for names of resources (such as myResourceGroup and vnetNetworkName). Note: There are certain resources, such as storage accounts, where the only option is to use lower case (and no other special characters). Consider using Azure Resource Manager policies (described in the next section) to enforce naming standards.  

Using Role Based Access Control (RBAC)

RBAC stands for Role Based Access Control where-by Azure provides various default roles for various namespaces and objects in each of these namespaces. The roles have a name and a set of allowed operations on those namespaces and as well as a set of operations that are not allowed. The general rule is anyone using Azure in any sort of capability should be in clear understood and well-defined roles which allow them to do only the specific activities they should be allowed to and nothing more and nothing less.  As a best practice every end user and administrator in Azure should only be limited to the set of roles that are absolutely required. Some of the following links are very handy in this regard.
  1. What is RBAC?
  2. Using RBAC to manage access to resources
  3. RBAC – Built-in roles
  4. RBAC – Custom roles
  5. Assign custom roles for internal and external users

Using Azure Policies

Azure Policy is another capability in the Azure platform where an enterprise can create various policies and then deploy them at an appropriate scope such as a subscription or a resource group. Based on the scope at which the policy is deployed, incoming ARM based requests to Azure get evaluated and ascertained if they conform to the policies. If they are not the incoming requests are not honored. The following are some handy links with regards to understanding and using Azure Policies.
  1. Overview of Azure Policies
  2. Azure Policy Definition structure
  3. Azure Policy Initiatives
 

Review Resource Group (RG) Costs periodically to mine opportunities for cost reduction

The best practice in terms of application deployment is to deploy the components of the solution all into a single Resource Group. The costs for each RG can be then separated out and analyzed for the following: - (a)    Relevancy – meaning is this component required any more. If not, then delete it. (b)    Usage and load patterns – if the usage is low or the load handled by the component is very low (or conversely high) then consolidate multiple instances of such components into lesser number of instances (or set scale up/down rules)

Making use of Tactical platform capabilities

Azure Hybrid User Benefit

One can save up to 40 percent on Windows Server virtual machines with Azure Hybrid Benefit for Windows Server. Use your on-premises Windows Server licenses with Software Assurance to save big on Azure. With this benefit, for each license we will cover the cost of the OS (on up to two virtual machines!), while you just pay for base compute costs. Overview Reference - https://azure.microsoft.com/en-us/pricing/hybrid-benefit/

Use Reserved Instances (RI)

As the name indicates Azure Reserved Instances are those VMs that you run, but which you have reserved for long term usage. The prices for Reserved VM Instances are significantly lower than non-reserved instances and so anytime there is a need to have long running VMs (say you are sure it is going to be used for a year or so) then you should opt to use Azure RIs to gain cost savings. Significantly reduce costs—up to 72 percent compared to pay-as-you-go prices—with one-year or three-year terms on Windows and Linux virtual machines (VMs). When you combine the cost savings gained from Azure RIs with the added value of the Azure Hybrid Benefit, you can save up to 82 percent*. Lower your total cost of ownership by combining RIs with pay-as-you-go prices to manage costs across predictable and variable workloads. What’s more, you can now improve budgeting and forecasting with a single upfront payment, making it easy to calculate your investments. Prepay for Virtual Machines with Reserved VM Instances The following are some useful references. To learn more about Reserved Virtual Machine Instances, see the following articles.  

Shutdown

Of all the strategies for saving costs on compute, shutting down a VM when not needed is the most propitious one. There are various methods to accomplish this.
  1. Now there is a provision in the portal (or via ARM) to shut down a VM at a certain time every day. So, see if that will work in your scenario and just set it for the VMs where this can be applied.
  2. Shutdown a VM when a certain idle threshold is reached – say VM CPU is 85% idle or whatever is applicable to your specific application and its downtime requirements. Idle detection is part of the platform and alerts can be generated which can then be used to trigger automation to shut down the VM in question.

Right Sizing

When provisioning VMs for various application workloads it is worth spending time and effort on finding the correct metrics that the application under load generates. These are the CPU load, IO throughput, memory requirements, etc. It might take a day or two get this information by testing/loading the application prior to migrating it. This will pay rich dividends when you deploy to Azure as it will give the correct size when you come to choose the VM family and size in Azure instead of blindly choosing the size with no proper data behind it. A right sized VM also has the advantage of not having to constantly resizing it.

Azure Low Priority VMs

Low-priority VMs are allocated from our surplus compute capacity and are available for up to an 80% discount, enabling certain types of workloads to run for a significantly reduced cost or allowing you to do much more for the same cost. Hence, in addition to considering AHUB and RI, also consider using Low Priority VMs whenever it is appropriate. The tradeoff for using low-priority VMs is that those VMs may not be available to be allocated or may be preempted at any time, depending on available capacity. The following links provide additional information on Low Priority VMs. Low Priority VMs – Overview Using Low Priority VMs in Batch Low-priority VMs are offered at a significantly reduced price compared with dedicated VMs. For pricing details, see Batch Pricing.

Scaling (Auto)

The Azure platform has baked in elastic and dynamic scaling that enable your azure resources such as compute to be dialed UP and DOWN in an automated fashion. The recommendation here is to make use of this platform capability in your solution deployments to scale UP and DOWN on demand, instead of using fixed size compute resources. As a prelude to using Auto Scaling for your compute deployments, always deploy VMs into Availability sets or VM Scale Sets. Once you deploy the VMs into these platform constructs leverage the auto-scale capabilities in here. If you are deploying applications into VMs, a la the old-fashioned IaaS method, then the least the application must satisfy is, is that it must be a stateless application. The following links provide various Scaling options. Azure Platform Auto-Scaling capabilities Using Azure Automation for Scaling VMs VM Scale Set Overview Scaling Low Priority VMs    

Performance optimization/performance POV

Choose the right storage

This was already covered in detail in the documentation on Storage Optimization. However briefly Azure provides the following types of storage capabilities from a replication perspective. (a)    LRS – Locally redundant storage (b)    ZRS – Zone Redundant storage (c)     GRS – Geographically Redundant storage (d)    RA-GRS Read Access – GA Choose the appropriate storage based on application replication needs. Obviously, GRS types of storage costs slightly more. Additionally, there are the options of (a)    HDD [spinning old style disks] – cost effective choice for low throughput non-mission critical apps. (b)    SDD [solid state drives] – Premium disks for high throughput I/O (c)     Managed Disks – it is recommended all production and critical applications use managed disks. Finally, the general recommendation is to use Managed Storage disks for all new VMs as they have significant advantages over non-managed disks and as well guaranteed SLAs besides not having to manage them as Azure manages them for the customer.

Choose the right type of VMs

Once the application team has decided that they would use IaaS based solution, landing on VMs, then spending time and deciding on the correct VM family and size will prove beneficial. There are several approaches to do this, some of which are outlined.
  1. For pure Lift and Shift type of scenarios, the current on-Prem deployment would serve as a starting baseline. Once you have the baseline, it is incumbent on each application team to profile the system characteristics under load. Once the load characteristics are known this can be mapped to the appropriate Azure VM Family and Azure Compute Units required to run the same application in Azure on VMs.

Choosing the appropriate platform construct

Many customers fall into this category where they do not have the time to understand the cloud native constructs provided by Azure and go for the easy pattern of lift and shift. While this is an easy mental model for experienced IT staff, it is not in tune with the various PaaS, SaaS and identity models that azure offers. This approach will almost invariably end up with choosing the wrong model and often advised by infrastructure-background personnel, who do not have the background to understand the developer capabilities of the platform, but who tend to push the discussion to the legacy approaches of deploying to VMs. From a cloud perspective these legacy approaches are rapidly become obsolete and as well as from an evolving software architecture point of view. This is a trap that should be avoided, and application teams should take the help of seasoned application developers who have kept abreast of various cloud native constructs which are baked right into the platform. The cost savings of such an approach are immense. Some of the benefits of microservices and cloud native capabilities are listed below. (a)    Overall savings from blindly spinning up VMs – this is an old model which should be *really* be the last resort. Cost of managing the IaaS based deployments as well as constantly monitoring, patching, backing up and taking care of the environment. Also, the lack of auto-scaling which will not enable cost realizations. (b)    Several hundreds to thousands of hours of savings for the development and deployment team as the cloud native constructs are built for rapid agile iterative cycles and super smooth DevOPS style deployment. None of these will be realized in the old-fashioned legacy way of deploying into VMS, by still wrongly adopting the technique of deploying into a VM/Server. (c)     Look for cloud native constructs the platform provides – a few examples
  1. Use of Cloud neutral and OS neutral capabilities like Service Fabric (SF)
  2. If SF is not feasible deploy the application into containers
  3. Serverless constructs like Azure Functions
(d)    Use PaaS capabilities such as the following: -
  1. Azure Web Applications
  2. Azure Logic Apps
  3. Azure Event Hubs
  4. Azure Service Bus
  5. Azure Event Grid
Most enterprise message bus type applications can be implemented by a combination of Azure Logic Apps, Azure Functions (or Web Jobs) with durable functions and storage. Such decoupling will lead to significant cost savings over and above the existing legacy architectures, besides being more scalable and easy to manage, resulting in less overall costs. Application reviews done so far point to the fact that customers are still following the older IT model of deploying into servers and this is a serious issue from a software architecture perspective. Such teams end up carrying huge technical debt, and will have to undergo a painful refactoring later to modern software architectures which exploit cloud native capabilities, in order to realize the resultant benefits of rapid DevOPS, huge time-savings, cost benefits, agile development cycles, besides having a decoupled and a highly decomposed(componentized)  architecture, which allows for independent versioning and management (of the different components that make the) of the  solutions/applications that need to continually manage, evolve and operate.  

REACTIVE Approach - things to do post deployment

Once the customer has started deploying into Azure then there are certain things that should be done post facto, in a continuous basis to wring out more cost efficiencies. These are spelt out below.

Azure Advisor

Turn on tools like Azure Advisor and implement all of its cost recommendations. The following link provides an intro to Azure Advisor which is a very simple tool to turn and as well as to implement its recommendations. https://docs.microsoft.com/en-us/azure/advisor/advisor-overview

Cloudyn type of tools

Cloudyn is another tool that can be used to study the usage and cost benefit analysis and it is recommended to turn this on and study its findings and recommendations and then implement them. Note that both Cloudyn and Azure Advisor use heuristics and AI to figure out usage patterns and make recommendations. So, the more these tools are turned on and have access to data over longer time periods their recommendations will make sense. The following link provides information on how to view Cost data with Cloudyn. https://docs.microsoft.com/en-us/azure/cost-management/quick-register-azure-sub  

Check if deployments can be converted for Reserved Instance(RI)

After having deployed into Azure it is often the case that customers retroactively try to make sure they are exploiting Azure RI wherever they can. So, it is incumbent on Azure administrators or those responsible for the apps and its associated costs to periodically scan the usage data and figure out which are all the Azure VM instances that RI can applied to.

Check if deployments can be converted for AHUB

The same applies to leveraging existing enterprise licenses for Windows Servers which the organization has already paid for. For all Windows VMs the AHUB benefits should be turned on.  

Study the entities within the RG

This is an activity that should be done periodically ideally atleast every couple of months by each application teams as part of the sprint cycle. The suggestion is to review the usage and cost data for every resource group in the monthly consumption data. By picking the top ten spenders in each RG and focusing on them and studying them and figuring out optimizations of these TOP TEN spenders, the following can be realized. Cost savings if any can be affected
  1. Valuable skills will be gained over a period of time by doing this exercise repeatedly.
  2. Rinse, repeat this atleast every 2 months.
 

Azure Compute VMs – Capability and Design Guidance

The sizing and tiering options provide customers with a consistent set of compute sizing options, which expand as time goes on. From a sizing perspective, each sizing series represents various properties, such as:
  • Number of CPUs
  • Memory allocated to each virtual machine
  • Temporary local storage
  • Allocated bandwidth for the virtual machines
  • Maximum data disks
  • GPU availability

VM Capability Guidance

  Some virtual machine series includes the concept of Basic and Standard tiers. A Basic tier virtual machine is only available on A0-A4 instances, and a Standard tier virtual machine is available on all size instances. Virtual machines that are available in the Basic tier are provided at a reduced cost and carry slightly less functionality than those offered at the Standard tier. This includes the following areas:
Capability Consideration Capability Decision Points
CPU Standard tier virtual machines are expected to have slightly better CPU performance than Basic tier virtual machines
Disk Data disk IOPS for Basic tier virtual machines is 300 IOPS, which is slightly lower than Standard tier virtual machines (which have 500 IOPS data disks).
Features Basic tier virtual machines do not support features such as load balancing or auto-scaling.
The following table is provided to illustrate a summary of key decision points when using Basic tier virtual machines:
Size Available CPU Cores Available Memory Available Disk Sizes Maximum Data Disks Maximum IOPS
Basic_A0 – Basic_A4 1 – 8 768 MB – 14 GB Operating system = 1023 GB Temporary = 20 - 240 GB 1 - 16 300 IOPS per disk
  In comparison, Standard tier virtual machines are available for all compute sizes.
Capability Consideration Capability Decision Points
CPU Standard tier virtual machines have better CPU performance than Basic tier virtual machines.
Disk Data disk IOPS for Basic tier virtual machines is 500. (This is higher than Basic tier virtual machines, which have 300 IOPS data disks.) If the DS series is selected, IOPS start at 3200.
Availability Standard tier virtual machines are available on all size instances.
A-Series features ·       Standard tier virtual machines include load balancing and auto-scaling. ·       For A8, A9, A10, and A11 instances, hardware is designed and optimized for compute and network intensive applications including high-performance computing (HPC) cluster applications, modeling, and simulations. ·       A8 and A9 instances have the ability to communicate over a low-latency, high-throughput network in Azure, which is based on remote direct memory access (RDMA) technology. This boosts performance for parallel Message Passing Interface (MPI) applications. (RDMA access is currently supported only for cloud services and Windows Server-based virtual machines.) ·       A10 and A11 instances are designed for HPC applications that do not require constant and low-latency communication between nodes (also known as parametric or embarrassingly parallel applications). The A10 and A11 instances have the same performance optimizations and specifications as the A8 and A9 instances. However, they do not include access to the RDMA network in Azure.
Av2-Series features ·       Represent new version of A-Series VM’s with amount of RAM per vCPU raised from 1.75 GB or 7 GB of RAM per vCPU to 2 GB or 8 GB per vCPU.  Local disk random IOPS has been improved to be 2-10x faster than that of existing A version 1 sizes.
D-Series features ·       Standard tier virtual machines include load balancing and auto-scaling. ·       D-series virtual machines are designed to run applications that demand higher compute power and temporary disk performance. D-series virtual machines provide faster processors, a higher memory-to-core ratio, and a solid-state drive (SSD) for the temporary disk.
Dv2-Series features ·       Standard tier virtual machines include load balancing and auto-scaling. ·       Dv2-series, a follow-on to the original D-series, features a more powerful CPU. The Dv2-series CPU is about 35% faster than the D-series CPU. It is based on the latest generation 2.4 GHz Intel Xeon® E5-2673 v3 (Haswell) processor, and with the Intel Turbo Boost Technology 2.0, can go up to 3.2 GHz. The Dv2-series has the same memory and disk configurations as the D-series.
Dv3-Series features ·       Standard tier virtual machines include load balancing and auto-scaling. ·       With Dv3-series, a follow-on to the original D/Dv2-series, Microsoft is introducing a new generation of Hyper-Threading Technology virtual machines for general purpose workloads
Ev3-Series features   ·       Standard tier virtual machines include load balancing and auto-scaling. ·       A new family for memory optimized workloads - introducing sizes with 64 vCPUs on Intel® Broadwell E5-2673 v4 2.3 processor and with 432 GB of memory on the largest Ev3 sizes
DS-Series features ·       Standard tier virtual machines include load balancing and auto-scaling. ·       DS-series virtual machines can use premium storage, which provides high-performance and low-latency storage for I/O intensive workloads. It uses solid-state drives (SSDs) to host a virtual machine’s disks and offers a local SSD disk cache. Currently, premium storage is only available in certain regions. ·       The maximum input/output operations per second (IOPS) and throughput (bandwidth) possible with a DS series virtual machine is affected by the size of the disk.
F-Series features ·       SKU is based on the 2.4 GHz Intel Xeon® E5-2673 v3 (Haswell) processor, which can achieve clock speeds as high as 3.1 GHz with the Intel Turbo Boost Technology 2.0. Having the same CPU performance as the Dv2-series of VMs, at a lower per-hour list price, the F-series is the best value in price-performance in the Azure portfolio based on the Azure Compute Unit (ACU) per core. The F-Series VMs are an excellent choice for gaming servers, web servers and batch processing. Any workload which does not need as much memory or local SSD per CPU core will benefit from the value of the F-Series.
FS-Series features ·       FS-series virtual machines can use premium storage, which provides high-performance and low-latency storage for I/O intensive workloads. It uses solid-state drives (SSDs) to host a virtual machine’s disks and offers a local SSD disk cache. Currently, premium storage is only available in certain regions. ·       The maximum input/output operations per second (IOPS) and throughput (bandwidth) possible with a DS series virtual machine is affected by the size of the disk.
G-Series features ·       Standard tier virtual machines include load balancing and auto-scaling. ·       Leverages local SSD disks to provide the highest performance virtual machine series that is available in Azure.
GS-Series features ·       Standard tier virtual machines include load balancing and auto-scaling. ·       Leverages local SSD disks to provide the highest performance virtual machine series that is available in Azure. ·       GS-series virtual machines can use premium storage, which provides high-performance and low-latency storage for I/O intensive workloads. It uses solid-state drives (SSDs) to host a virtual machine’s disks and offers a local SSD disk cache. Currently, premium storage is only available in certain regions. ·       The maximum input/output operations per second (IOPS) and throughput (bandwidth) possible with a GS series virtual machine is affected by the size of the disk.
H-Series features ·       H-series VMs are based on Intel E5-2667 V3 3.2 GHz (with turbo up to 3.5 GHz) processor technology, utilizing DDR4 memory and SSD-based local storage.  The new H-series VMs furthermore features a dedicated RDMA backend network enabled by FDR InfiniBand network, capable of delivering ultra-low latency.  RDMA networking is dedicated for MPI (Message Passing Interface) traffic when running tightly coupled applications. ·       Provide great performance for HPC applications in Azure.  H-series VM sizes is an excellent fit for any compute-intensive workload.  They are designed to deliver cutting edge performance for complex engineering and scientific workloads like computational fluid dynamics, crash simulations, seismic exploration, and weather forecasting simulations.
N-Series features A new family of Azure Virtual Machines with GPU capabilities suited for compute and graphics-intensive workloads – aiding in scenarios like remote visualization, high performance computing and analytics. To be available in preview in Q1CY16 the N-series will be based on NVidia’s M60 and K80 GPUs and will feature the NVIDIA Tesla Accelerated Computing Platform as well as NVIDIA GRID 2.0 technology, providing the highest-end graphics support available in the cloud today
    The following summary of the capabilities of each virtual machine series is provided in the following table:  
Size Available CPU Cores Available Memory Available Disk Sizes Maximum Data Disks  Maximum IOPS
Basic_A0 – Basic_A4 1 – 8 768 MB – 14 GB Operating system = 1023 GB Temporary = 20-240 GB 1 – 16 300 IOPS per disk
Standard_A0 – Standard_A11 (Includes compute intensive A8-11) 1 - 16 768 MB - 112 GB Operating system = 1023 GB Temporary = 20-382 GB   1 – 16 500 IOPS per disk
Standard_A1_v2 – Standard_A8_v2; Compute-intensive Standard_A2m_v2 - Standard_A8m_v2 1 - 8 2 GB – 64 GB Operating system = 1023 GB Temporary SSD disk = 10-80 GB 1 – 16 500 IOPS per disk
Standard_D1-D4 Standard_D11-D14 (High memory) 1 - 16 3.5 GB – 112 GB Operating system = 1023 GB Temporary (SSD) =50 – 800 GB 2 – 32 500 IOPS per disk
Standard_D1v2-D5v2 Standard_D11v2-D14v2 (High memory, faster CPU) 1 - 16 3.5 GB – 112 GB Operating system = 1023 GB Temporary (SSD) =50 – 800 GB 2 – 32 500 IOPS per disk
Standard_DS1-DS4 Standard_DS11-DS14 (Premium storage) 1 - 16 3.5 – 112 GB Operating system = 1023 GB Local SSD disk = 7 GB – 112 GB GB 2 – 32 43 – 576 GB cache size 3200-50000 IOPS total
Standard_F1 – F16   1-16 2-32 Local SSD disk: 16 GB – 256 GB 2-32 500 IOPS per disk
Standard_FS1 – FS16 1-16 2-32 Local SSD disk: 4GB – 64GB 2-32 4,000-64,000 IOPS
Standard_G1 – G5 (High performance) 2 - 32 28 GB – 448 GB Operating system = 1023 GB Local SSD disk = 384 – 6,144 GB 4 – 64 500 IOPS per disk
Standard_GS1 – GS5 (High performance, Premium storage) 2 - 32 28 GB – 448 GB Operating system = 1023 GB Local SSD disk = 56 – 896 GB 4 - 64 264 – 4224 GB cache size 5000-80000 IOPS total
Standard_H8 - H16 8-16 56 GB - 224 GB Local SSD Disk: 1,000-2,000 GB 16 - 32 500 IOPS per disk
      The table below also describes differences between A-series and Av2-series  
SIZE vCPU RAM (GiB) Disk Size   Size vCPU RAM (GiB) Disk Size
A1 1 1.75 20 GB (HDD) → A1_v2 1 2 10 GB (SSD)
A2 2 3.50 70 GB (HDD) → A2_v2 2 4 20 GB (SSD)
A3 4 7 285 GB (HDD) → A4_v2 4 8 40 GB (SSD)
A4 8 14 605 GB (HDD) → A8_v2 8 16 80 GB (SSD)
A5 2 14 135 GB (HDD) → A2m_v2 2 16 20 GB (SSD)
A6 4 26 285 GB (HDD) → A4m_v2 4 32 40 GB (SSD)
A7 8 52 605 GB (HDD) → A8m_v2 8 64 80 GB (SSD)
  These sizing and capabilities are for the current Azure Virtual Machines, and they might expand over time. For a complete list of size tables to help you configure your virtual machines, please see: Sizes for Virtual Machines.  

Azure VM Design Guidance

 
Design Guidance
  When you design solutions for using virtual machines, consider the following:
Capability Considerations Capability Decision Points
Deployment order If you intend to deploy an application that may require compute intensive resources, it is recommended that customers provision a virtual machine to a cloud service with the largest virtual machine (such as Standard_G5) and scale it down to a more appropriate size. The reason is that virtual machines will be placed on the clusters that have the faster processors. It also makes scaling easier and it is more efficient to combine resources.
Supportability The following are not supported in a virtual machine on Microsoft Azure: ·       Multicast ·       Layer-2 routing ·       32-bit OS versions ·       Windows Server OS versions prior to Windows Server 2008 R2 Note: Windows Server 2003 / 2008 32-bit OS versions are supported for deployment on Azure virtual machines with substantial limitations, including no support for agents or extensions.
   

Usage Analysis Tools

The following tools are available as part of the platform and consider turning them on. (A)    Azure Advisor (B)    Azure Security Center (C)    Cloudyn They analyze the underlying data emitted by various Azure resources and arrive at recommendations from the perspective of (i)                  Cost (ii)                Security (iii)               Performance In most cases these recommendations are implementable in an automated fashion, if not via a single click. One can also review and implement the recommendations manually if one prefers to do that.      ]]> Logical Volume Manager(LVM) with RedHat RHEL VMs in Azure and custom partitioning https://docs.microsoft.com/archive/blogs/girishp/logical-volume-managerlvm-with-redhat-rhel-vms-in-azure-and-custom-partitioning Thu, 23 Nov 2017 23:03:33 GMT https://blogs.msdn.microsoft.com/girishp/?p=1925 For RHEL images with LVM support Use the following in your VM creation parameters.
  1. Publisher = "RedHat"
  2. SKU = "7-LVM"
  3. Offer = "RHEL
Now specifically in your arm templates in the storage profile you would refer to them as follows:- "imagePublisher": "RedHat",     "imageSku": "7-LVM",     "imageOffer": "RHEL", You could take a simple OOB RHEL template already present in the Azure GitHub Quick Start template and make the necessary modifications and create this image. To start simply take the following template - https://github.com/Azure/azure-quickstart-templates/tree/master/101-vm-simple-rhel  and then modify it in the variables section as shown above and deploy the image using PowerShell or Visual Studio. If you wanted to try a more complex RHEL based template try the following by using the same changes as above. 3 Tier RHEL based deployment   Once the VMs have been created you can login and do the custom partition that you need. If you want to automate the partition then several methods are available. (1) Using Linux Agent Linux Custom Script Extensions OR (2) Using other tools like Ansible.]]> Continuous Integration and Continuous Delivery (CI/CD) https://docs.microsoft.com/archive/blogs/girishp/continuous-integration-and-continuous-delivery-cicd Mon, 20 Nov 2017 18:58:53 GMT https://blogs.msdn.microsoft.com/girishp/?p=1835 The following is a collection of recent capabilities announced by Microsoft in the area of CI/CD.

Continuous Integration & Continuous Delivery

 

Deploy your app to Linux VMs by using Jenkins and Team Services

Approvals and gates overview

Approvals

Build definition source repositories

Define a continuous integration (CI) build process for your GitHub repo

Define a continuous integration (CI) build process for your Git repo

How to use YAML builds

 

Utility Tasks

Utility task: Manual intervention

Utility task: Azure function

Utility task: Azure monitor

Utility task: Invoke HTTP REST API

Utility task: Query Work Items

Install and license Package Management

Utility task: Publish Build Artifacts

Utility task: Copy and Publish Build Artifacts

Variable groups

Microsoft Connect 2017 – DevOps project right in Azure etc.

Cloud-hosted Mac agents for CI/CD pipelines

Release Gates: Releases with continuous monitoring built in

Introducing Azure DevOps Project

Announcing “Azure DevOps Project” public preview

Pipeline as code (YAML) preview

While You’re Waiting for Connect(), Check out DevOps at Microsoft

 

 ]]> Different ways of slicing and dicing OMS Log Data in Azure - VM Creation https://docs.microsoft.com/archive/blogs/girishp/different-ways-of-slicing-and-dicing-oms-log-data-in-azure-vm-creation Tue, 14 Nov 2017 21:39:34 GMT https://blogs.msdn.microsoft.com/girishp/?p=1736 [Please click on pic to get a clearer version] The below is an example of a simple requirement to list all VM Created, along with who created them and when they were created. I outline two methods below. Pick and choose what is appropriate for your needs.

Method 1

Using the Activity Logs for the subscriptions from the Azure Portal

 
  1. Go to "More Services" as shown below.
[Please click on pic to get a clearer version]  
  1. On the Search pane type "Log" and choose Activity Logs
  [Please click on pic to get a clearer version]   Configure the Query parameters as follows:- [caption id="attachment_1756" align="alignnone" width="300"] Activity Log Query [Please click on pic to get a clearer version][/caption]     

Method 2

 

Using OMS Log Analytics

For this one has to have an OMS workspace associated to the subscription.

QUERY TEXT

  AzureActivity | where OperationName endswith "Write" | where ActivityStatus == "Succeeded" | where ActivitySubstatus contains "Created" | project resourceName=Resource, CreatedBy=Caller, dateTimeOfCreation=TimeGenerated, ResourceGroup, SubscriptionId, ResourceId [caption id="attachment_1765" align="aligncenter" width="734"] OMS Log Analytics Query [Please click on pic to get a clearer version][/caption]There are various schemas you can check to see if it is the one that contains the data that you want to query on. A sample of these schemas is shown below. [caption id="attachment_1785" align="alignnone" width="300"] Various Schemas[/caption]            ]]> ARM Template - combining Application Gateway and enabling diagnostic logging into OMS https://docs.microsoft.com/archive/blogs/girishp/arm-template-combining-application-gateway-and-enabling-diagnostic-logging Tue, 09 May 2017 03:22:38 GMT https://blogs.msdn.microsoft.com/girishp/?p=1695 Introduction Often times you are looking to go beyond the basic ARM template and as a Security/Infrastructure admin for the CORP's Azure deployment, your objective is to make sure Applications being deployed are following best practices some of which are:- (a) Applications should be behind an Application Gateway (AppGW) combined/protected by a Web Application Firewall(WAF). (b) Application should be funneling their diagnostic logs to an Operations Manager sink. (c) Maybe other things from a security view point you can add. Requirement Application developers will deploy their applications and have infrastructure Code in the form of JSON templates OR the application owners hand over the Application code binaries and the infrastructure environment in the form of ARM template. Now you, the Azure Administrator, is responsible for securely deploying this application into the PROD environment, with the added requirements of Application security and logging. And you need to repeat this multiple times for multiple applications - so you need a template that combines the basic infrastructure the Application requires, plus a template to provision AppGW/WAF and combine with turning on diagnostic log and feed it into an OMS sink. Implementation You could use different techniques to do this. The first one that comes to mind is to create this from the portal in Azure and then reverse engineer the template out of the deployment. Unfortunately you cannot do that as of today (05/2017) as AppGW schema is not supported in the "Automation Script" - you will get the following error, when you try this reverse engineering route. Application gateways cannot be exported yet and is not included in the template. See error details. The schema of resource type 'Microsoft.Network/applicationGateways' is not available. Resources of this type will not be exported to the template. (Code: ResourceTypeSchemaNotFound, Target: Microsoft.Network/applicationGateways) So we are left with the taking AppGW template from one of the public repository and then combining it appropriately with the requisite JSON fragments for diagnostic settings, testing it, fixing errors till the deployment runs through completion. Then verify the actual deployment of the AppGW and the diagnostic settings persisted in the portal and as well as logs show up in the OMS sink. Process I essentially needed an AppGW template to start with. For this I used one from this link Application Gateway Templates Also the second ingredient required is the method for the diagnostic settings. The reference to this can be found in Templatizing diagnostic log settings First fire up Visual Studio and then create a project of type "Azure Resource Group". Use a blank template and copy over one of your chosen AppGW based template. You could use one from the first link above OR use any other template you have for an Application Gateway. vs-2017-05-08_22-02-37   Let me highlight some sections, that I had to figure out to get this to work. The important thing is to add the resource specification for the diagnostic settings (in the second link above) correctly as a nested resource to the Application Gateway resource. This is shown in the below graphic. Click on it to see a clearer view Nested resource for Diagnostics under an App-GW As you can see in the graphic above, I have the primary resource "Application Gateway" and under it a nested resource of type "providers/diagnosticSettings" which depends on a resource of type "Microsoft.Network/applicationGateways". "type": "providers/diagnosticSettings", "name": "Microsoft.Insights/service", "dependsOn": [ "[concat('Microsoft.Network/applicationGateways/', variables('applicationGatewayName'))]" ], Parameter File In my parameter file I already created the storage account and workspaceID which I then referred directly below. You could also use the JSON "depends" notation and create them dynamically if you so desired. In my case I chose to precreate them and refer them as parameters in my template, as I was mainly testing "HOW TO: Add the diagnostic log settings for an OMS sink into the main AppGW template. { "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentParameters.json#", "contentVersion": "1.0.0.0", "parameters": { "backendIpAddress1": { "value": "appGWIP1" }, "backendIpAddress2": { "value": "appGPIP2" }, "storageAccountName": { "value": "myteststore321" }, "workspaceId": { "value": "/subscriptions/xxxx8775-d18a-45bb-abcd-3627de55xxxx/resourcegroups/mytest/providers/microsoft.operationalinsights/workspaces/mytestoms" } } } The full form of the main template is posted in the graphic below. To see it clearly click on the image and then hit "Ctrl +" in the browser Full Main Template]]> MAS - Azure Stack Infrastructure Roles https://docs.microsoft.com/archive/blogs/girishp/mas-azure-stack-infrastructure-roles Tue, 25 Apr 2017 19:47:06 GMT https://blogs.msdn.microsoft.com/girishp/?p=1676 Note Everything is with respect to a point in time. The point in time reference to this is TP3 Refresh timeline, though there may be changes later. Infrastructure Roles A picture is worth a thousand words. The following is a list of the infrastructure roles available when you deploy TP3 refresh. Click to see expanded/clear view. [caption id="attachment_1686" align="alignnone" width="300"]Infrastructure Roles Infrastructure Roles [/caption] The following are a few brief lines of what each one does in addition to several other things they do, that are not exposed. 1. Network Controller - As the name indicates the Network Controller provides network services and is the underlying the controller the Network Resource Provider (RP) talks to, when users submits requests to that RP. 2. Compute Controller - Again as the name indicates the Compute RP needs to talk to this controller before any Compute related resources such as VMs, VMSSs, VMExtensions etc are provisioned. 3. Storage Controller - The controller that Admins/End users need to route their requests to, to provision anything storage related. Also it does all the magic under the covers of Storage Spaces direct, and taking care of disk and data resiliency. 4. Backup Controller - This is an additional service engineered by the MAS team, to do the backup of the infrastructure related data onto a file share in the customers' data center. Note that for redundancy purposes, one would funnel this backup data to another location also, once the backup controller backs up the infrastructure data & meta-data onto an on-prem file share. 5. Edge Gateway - a single node MAS installation creates an Edge Gateway, through which traffic is routed IN and OUT of the solution. 6. Load Balancer MUX - At a high level, this does the magic of doing the network translation and as well as routing the packets inside the solution. Simply put the SLB MUX load balancer works just like in public Azure to publish the RDP port and any other ports you choose to open up. It provides Load Balancing, NAT'ting, route table propagation (using BGP), along with the enforcing NSG rules - all in one solution - it does the real SDN magic. 7. ADFS - MAS comes with an built in ADFS role. Its primary purpose is if you wanted to operate with direct access to an on-prem AD, then you can use the MAS built-in ADFS to federate with an on-prem AD. 8. Infrastructure Deployment - This is the role that takes care of installing, updating, patching etc.,, of your MAS deployment - aka the ECE component. 9. Health Controller - As the name indicated this does the collection and probing for Health of the system, and as well as surfacing the Health events back to the Health RP and via it to external systems like the portal OR to PS, and APIs who query for health events. 10. ARM - This is the role that sits on top of all of the RPs and provides a consistent management and provisioning layer, allowing life-cycle (CRUD) activities of resources via ARM templates, and mediates and distributes all of the incoming RP requests from the Admins and tenant users alike. 11. Cert Management - MAS uses its own Certificate manager for various things among which it also validates inter-role, inter-RP and other internal communications. It also rotates/expires the certificate aggressively to prevent any tampering with the secured communications that happens inside this sealed system. 12. ACS Ring This provides among various other things, the storage health and life-cycle management of storage related components, and built using the Service Fabric Ring architecture, due to its need to be highly resilient and fault tolerant, to keep the underlying storage sub-systems and data integral and available at all times. Again this does the software defined storage management under the covers, like gobbling up additional disk capacity into the storage cluster transparent to the end users. PS This information is from deploying the publicly available TP3 refresh bits and from publicly available data/documentation. ]]>