• Git client
• Code editor such as Visual Studio Code
• Python 2.7

Setting-up Application Insights

• Login to Azure Portal then click on Create a resource
• Click on Development tools, then select Application Insights
• Enter the app insights details as shown below then click create

      

• Once the resource is created, go to AppInsightsRG and click on pythonapp.
• In Essentials copy the Instrumentation Key and keep it handy.  We will need that later when we start instrumenting our code.

Downloading the Sample Application

python .\runserver.py

Instrumenting your Python Application

pip install applicationinsights

from applicationinsights.flask.ext import AppInsights
app.config['APPINSIGHTS_INSTRUMENTATIONKEY'] = '<YOUR INSTRUMENTATION KEY GOES HERE>'
# log requests, traces and exceptions to the Application Insights service
appinsights = AppInsights(app)

<script type="text/javascript">
var appInsights=window.appInsights||function(config){
function s(config){t[config]=function(){var i=arguments;t.queue.push(function(){t[config].apply(t,i)})}}var t={config:config},r=document,f=window,e="script",o=r.createElement(e),i,u;for(o.src=config.url||"//az416426.vo.msecnd.net/scripts/a/ai.0.js",r.getElementsByTagName(e)[0].parentNode.appendChild(o),t.cookie=r.cookie,t.queue=[],i=["Event","Exception","Metric","PageView","Trace"];i.length;)s("track"+i.pop());return config.disableExceptionTracking||(i="onerror",s("_"+i),u=f[i],f[i]=function(config,r,f,e,o){var s=u&&u(config,r,f,e,o);return s!==!0&&t["_"+i](config,r,f,e,o),s}),t}({
instrumentationKey: "<YOUR INSTRUMENTATION KEY GOES HERE>"
});
window.appInsights=appInsights;
appInsights.trackPageView();
</script>

Sending Custom Events

@app.route('/track_custom_event')
def track_custom_event():
    from applicationinsights import TelemetryClient
    tc = TelemetryClient(app.config['APPINSIGHTS_INSTRUMENTATIONKEY'])
    tc.track_event('ShoppingCartCheckout')
    tc.flush()
    return render_template('index.html')

<div class="row">
  <div class="col-md-4">
     <h2>Send Custom Event</h2>
     <p>Press button to send custom event called <b>ShoppingCartCheckout</b></p>
     <p><a class="btn btn-default" href="{{ url_for('track_custom_event') }}">Custom Event &raquo;</a></p>
  </div>
</div>

Sending Custom Metrics

@app.route('/send_metric')
def send_metric():
    from applicationinsights import TelemetryClient
    tc = TelemetryClient(app.config['APPINSIGHTS_INSTRUMENTATIONKEY'])
    import random
    tc.track_metric('queueSize',  random.randint(0,5000))
    tc.flush()
    return render_template('index.html')

<div class="col-md-4">
   <h2>Send Custom Metric</h2>
   <p>Press button to send custom metric for <b>queueSize</b></p>
   <p><a class="btn btn-default" href="{{ url_for('send_metric') }}">Custom Metric &raquo;</a></p>
</div>

Reporting Handled Exceptions

@app.route('/handledException')
def handledException():
    import sys
    import string
    import random
    from applicationinsights import TelemetryClient
    tc = TelemetryClient(app.config['APPINSIGHTS_INSTRUMENTATIONKEY'])
    try:
        randomstring = ''.join([random.choice(string.ascii_letters) for n in xrange(8)])
        raise Exception("Exception {0}".format(randomstring))
    except:
       tc.track_exception()
   
    try:
       randomstring = ''.join([random.choice(string.ascii_letters) for n in xrange(8)])
       raise Exception("Exception {0}".format(randomstring))
   except:
       tc.track_exception(*sys.exc_info(), properties={ 'foo': 'bar' }, measurements={ 'x': 42 })
       tc.flush()
   
   return render_template('index.html')

<div class="col-md-4">
   <h2>Trigger handled Exception</h2>
   <p>Press button to trigger a random handled exception</p>
   <p><a class="btn btn-default" href="{{ url_for('handledException') }}">Handled Exception &raquo;</a></p>
</div>
 

Wrap-up

• Database username and password or connection strings
• API keys
• Authentication (i.e. OAuth) tokens
• Third party service username and password
• Any other sensitive information your app might need

• After the config file is checked in to the version control system, the sensitive information would be exposed
• When information changes, a deploy might be required for the new changes to take effect
• Secrets for production environment will need to be stored somewhere where they are properly protected. Sometimes that’s left to the judgement of the secrets maintainer, which might result in secrets being compromised if they are not kept in a secured location
• Maintaining lifecycle of these secrets might not be easy since they might be scattered all over the place

Approach Overview

Enable your web app to communicate with Azure Key Vault

Create a sample project

1. From Visual Studio go to File => New => Project
2. Select Template => Web => NET Web Application (.NET Framework) and click Ok
3. Select MVC and click Ok

Register app with Azure Active Directory

Enable Application to Communicate with Azure Active Directory and Key Vault

1. Microsoft.IdentityModel.Clients.ActiveDirectory
2. Microsoft.Azure.KeyVault

<add key ="ClientId" value=""/>
<add key="ClientSecret" value="" />
<add key="SecretUri" value="" />

using Microsoft.IdentityModel.Clients.ActiveDirectory;
using System;
using System.Threading.Tasks;
using System.Web.Configuration;

namespace WebApplication1.Util
{
   public class KeyVaultHelper{
      public static string MySecret { get; set; }
      public static async Task<string> GetToken(string authority, string resource, string scope){
         var authContext = new AuthenticationContext(authority);
         ClientCredential clientCred = new ClientCredential(WebConfigurationManager.AppSettings["ClientId"],
         WebConfigurationManager.AppSettings["ClientSecret"]);
         AuthenticationResult result = await authContext.AcquireTokenAsync(resource, clientCred);

         if (result == null)
            throw new InvalidOperationException("Failed to obtain the JWT token");
         return result.AccessToken;
      }
   }
}

var kv = new KeyVaultClient(new KeyVaultClient.AuthenticationCallback(KeyVaultHelper.GetToken));
var task = kv.GetSecretAsync(WebConfigurationManager.AppSettings["connectionString"]);
task.Wait();
KeyVaultHelper.connectionStringSecret = task.Result.Value;

public class ApplicationDbContext : IdentityDbContext
{
   public ApplicationDbContext()
      : base(dbConnectionString(), throwIfV1Schema: false)
   {

   }
   
   public static string dbConnectionString(){
      return KeyVaultHelper.EncryptSecret;
   }

   public static ApplicationDbContext Create()
   {
      return new ApplicationDbContext();
   }
}

1. Right click on your solution the click Add => New Project => Cloud => Azure Resource Group and give it a name
2. Right click on Scripts folder and select New Item then create a file called SetAzureWebsite.ps1 and set its content to the following:

param (
   [string] $AzureWebsiteName,
   [string] $slot,
   [hashtable] $appsettings
)
Set-AzureWebsite -Name $AzureWebsiteName -Slot $slot -AppSettings $appsettings

• Web App
• Database Server
• Database

Using Key Vault to Store Secrets

1. Navigate to https://portal.azure.com
2. Create resource group called mydemoRG
3. Go to mydemoRG and create Key Vault and give a unique name. In my case I named it  
4. Once created, click on it and go to Overview => Secrets => Add
5. for upload option choose Manual and for name enter connectionstring and for value field set it to the connection string of the database created earlier. To obtain the connectionString from database created earlier, click on the database and then go to Overview and then click on Show database connection strings.  Make sure {your_username} and {your_password} are replaced with the actual values for your environment before copying to the value of the secret.   Once done, you click create.
6. Go back to the key vault you created and click Overview then Principals
7. Click Add New
8. In Select principal type the name you registered earlier with Azure Active Directory
9. For Secret Permissions, check Get
10. Click Ok and then Save

Build and Deploy app using VSTS

/p:PackageLocation="$(build.sourcesdirectory)" /p:DeployOnBuild=true /p:WebPublishMethod=Package /p:PackageAsSingleFile=true /p:SkipInvalidConfigurations=true

-AzureWebsiteName yourwebappname -Slot production -appsettings @{"ClientId" = "your_client_id"; "ClientSecret" = "your_client_secret"; "connectionString" = "your_keyVault_connectionString_url"}

Creating Azure Storage Account

• Login to Azure portal and create an Azure Storage account following the steps documented here. Specify the parameters as shown below.  Make sure the Name is unique.

• Once deployment is done, go to the storage account and click on Shared access signature then click on Generate SAS and connection string. Copy the SAS token generated and keep it handy as we will use it later
• Go to the storage account Overview page and then click on Blobs
• Add a Container called “armartifacts” as shown below

• Once the container is created, click on it and go to Container properties. Copy the URL field and keep it handy.  We will need it later as well

Protecting Secrets with Azure Key Vault

• In the Azure portal, create an Azure Key Vault resource
• Click on the Azure Key Vault you just created and click on Secrets
• Click on Generate/Import to add the SAS Token
• For name, enter “StorageSASToken” and enter the Azure Storage shared access signature key you copied in a previous step to the Value field
• Click Create

Linking Azure Key Vault to VSTS

• Login to your VSTS account. If you don’t have one, you can create one for free here
• Go to Build and Release hub in VSTS and click Library
• In Variable Group name field enter “AzureKeyVaultSecrets”
• Toggle “Link secrets from an Azure key vault as variables”
• Select your Azure subscription and then the Azure key vault you created earlier and click Authorize
• Once authorization is successful you can add variables by clicking “Add” and you should be presented with the option to add references to the secrets in the Azure key vault. Once you added the reference to the StorageSASToken, click Save.

Set-up Continuous Deployment using VSTS

• To enable Continuous deployment, click on trigger button and toggle Continuous deployment trigger

 

• Click Variables then Variable groups then click on Link variable group. Select the AzureKeyVaultSecrets variable group that you created in an earlier step
• Add a variable called AzureBlobStorageURL and paste in the value from the url you copied earlier when you created the Azure Storage account
• Add a variable called blobContainerName and put the name of your Azure blob container
• Add a variable called StorageAccountName and put the name of your Azure storage account

• In the release tasks, create the following steps:

• Once steps are added, click save and kick-off a release. The release should finish successfully

• Check Azure and you should see the VNet with NSG created

• Containerization with Docker
• Cluster management and container orchestration using Kubernetes
• Team Foundation Server (TFS) or Visual Studio Team Services (VSTS)

• Visual Studio 2017 Enterprise Edition
• Visual Studio Team Services account. You can signup for a free VSTS account here

Setting up CI for an ASP.NET Core app using VSTS

Creating Sample app

Creating Continuous Integration Build using VSTS

Setting up OpenShift Environment

• Install the Openshift Origin CLI from here
• Start Poweshell or command prompt

oc login your_Openshift_url -u youruser -p yourpassword

• Ensure you are on the right project by running

oc projects

• If you are not pointing to sample-project

oc project sample-project

• Create a file called samplewebapp-oc-deploy.yml in your current directory and fill it with the following

apiVersion: v1
kind: DeploymentConfig
metadata:
  name: sampleapp-deployment
spec:
  replicas: 1
  template:
    metadata:
      labels:
        name: sampleapp-deployment
    spec:
      containers:
      - image: "yourrepo/samplewebapp:versionnumber"
        name: samplewebapp
        ports: 
        - containerPort: 80

• Deploy the app

oc apply -f samplewebapp-oc-deploy.yml

• Create a service for the deployment

oc expose deploymentconfig sampleapp-deployment  --port 80 --type=LoadBalancer

• Create a route to be able to browse to your app

oc create route edge --service sampleapp-deployment --port 80

• Retrieve the route that was just created

oc get routes

• browse to "https://your_url/api/values".  Make sure your_url is the url returned from previous step.  In my case, the complete address is https://sampleapp-deployment-sample-project.192.168.99.100.nip.io/api/values

Setting up Continuous Deployment

apiVersion: v1
kind: DeploymentConfig
metadata:
  name: sampleapp-deployment
spec:
  replicas: 1
  template:
    metadata:
      labels:
        name: sampleapp-deployment
    spec:
      containers:
      - image: #{IMAGETAG}#
        name: samplewebapp
        ports: 
        - containerPort: 80 

• Applications need to be developed, deployed and enhanced at a rapid pace
• Applications are always available, resilient and performant
• Features expected to match or exceed competitors
• Applications need to run on different form factors including PCs and mobile
• All applications delivered must be secure and compliant

External Facing Components

Web Portal

• Availability: users should be able to access the system at any time. The system should also be resilient to handle application as well as infrastructure failures.  Also, there should not be any single point of failure in the system.  If a component in the system fails, the system should continue to respond to requests that are targeting other components in the system
• Scalability: as the user base grows, the system should continue to honor its SLAs.
• Security: only authorized users should be allowed to login and access the system. Also data can only be accessed by users that have the proper permissions to view and manipulate it.
• Maintainability: because this component is the entry point to the system it is paramount that mechanisms be put in place to enable easy deploys and near-zero downtime updates

• Azure web App
• Traffic Manager
• Azure Service Fabric
• Azure Container Service
• Azure Active Directory (for authentication and authorization)
• Content Delivery Network (CDN)

Authentication and Authorization

• Multi-factor Authentication
• Self-service Password Management
• Role Based Access Control
• Application Usage Monitoring
• Rich auditing and Security Monitoring and Alerting

Content Delivery Network (CDN)

Service Gateway

• Expose all APIs behind a single static IP and domain
• Get near real-time usage, performance and health analytics
• Automate management and integrate using REST API, PowerShell, and Git
• Provision API Management and scale it on demand in one or more geographical regions
• Self-service API key management
• Auto-generated API catalog, documentation, and code samples
• OAuth-enabled API console for exploring APIs without writing code
• Sign in using popular Internet identity providers and Azure Active Directory
• Client certificate authentication
• Simplify and optimize requests and responses with transformation policies
• Secure APIs with key, JSON Web Token (JWT) validation, and IP filtering
• Protect your APIs from overload and overuse with quotas and rate limits
• Use response caching for improved latency and scale

Internal Facing Components

Domain Services

Service Fabric

Azure Container Service

Batch Processes

• Azure Batch: a platform service for running large-scale parallel and high-performance computing (HPC) applications efficiently in the cloud. Azure Batch schedules compute-intensive work to run on a managed collection of virtual machines, and can automatically scale compute resources to meet the needs of your jobs.
• Azure Functions: a Serverless technology that allows to run small pieces of code, in the cloud without worrying about a whole application or the infrastructure to run it.
• Logic App: provides a way to simplify and implement scalable integrations and workflows in the cloud. It provides a visual designer to model and automate your process as a series of steps known as a workflow.

Data Persistence Layer

Reporting

Pre-requisites

• Docker Integration
• Kubernetes extension

Setting up Continuous Integration

• Login to the Team Project in VSTS where the code is checked-in
• Tap Build and Release tab and then click on Builds
• Click New definition button
• Click on empty process link
• Add Docker Compose task from the Docker integration extension installed from Marketplace. Set Docker Compose File field to docker-compose.ci.build.yml.  Set Command field to up

 

• Add another Docker Compose task from the Docker integration extension installed. Set Docker Compose File field to docker-compose.yml.  Set Command field to build

 

• Add Docker step from the Docker integration extension. For Docker Registry Connection field, click the plus sign (+) and you will be prompted to enter your Docker registry credentials.  VSTS uses those credentials to push the generated Docker image.  Set Action field to Push an image.   Set image name to xyz/sampleapi:latest.  Make sure you replace “xyz” with your actual Docker repo name

• Add Copy and Publish Build Artifacts task and configure it as shown below
• Save and queue a build by clicking the Save & queue button. For the Agent queue field, select Hosted Linux Preview option

• Click Queue and the build should be queued and executed once an agent is available

Setting up Continuous Deployment

• Create a deploy file called yml with the content below and check it in to the repository:

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: sampleapi-deployment
spec:
  replicas: 3
  template:
    metadata:
      labels:
        app: sampleapi
    spec:
      containers:
      - name: sampleapi
        image: xyz/sampleapi:latest
        ports:
        - containerPort: 80

• Download kubectl from here and check it in to the repository
• Tab on Build & Release and click Releases
• Click on New Definition
• Select Empty as the release template and click Next

• Check Continuous Deployment option
• Click on Add Tasks
• Select Kubernetes Apply Task. Click on Add next to k8s end point field and fill out the information for your Kubernetes cluster with information similar to below

• For Kubeconfig field, set it to the content from the config file inside .kube directory generated when the az acs kubernetes get-credentials is run as show in the previous post

• Click OK to return to the task
• Set the YAML file and kubectl binary as shown below

• Click on Run on agent and make sure you select Hosted Linux Preview
• Click save icon and queue a release and this should conclude setting up Continuous Deployment.

• Ability to scale specific components of the system. If “Feature1” is experiencing an unexpected rise in traffic, that component could be scaled without the need to scale the other components which enhances efficiency of infrastructure use
• Improves fault isolation/resiliency: System can remain functional despite the failure of certain modules
• Easy to maintain: teams can work on system components independent of one another which enhances agility and accelerates value delivery
• Flexibility to use different technology stack for each component if desired
• Allows for container based deployment which optimizes components’ use of computing resources and streamlines deployment process

Creating a simple web API with Docker Support using Visual Studio

• In Visual Studio, go to File => New => Project => Web => ASP.NET Core Web Application (.NET Core)

• Select Web API and make sure that Enable Docker Support is enabled and then click OK to generate the code for the web API

• Open docker-compose.yml file and prefix the image name with your docker repository name. For example, if your Docker repository name is xyz then the entry in the docker-compose.yml should be:  image: xyz/sampleapi  Here is how that file should look like:

version: '1'
services:
  sampleapi:
    image: xyz/sampleapi
    build:
      context: ./SampleAPI
      dockerfile: Dockerfile

• Check in the generated Web API code into Version Control System. You can use the git repository within Visual Studio Team Services for free here
• Checkout the code to an Ubuntu machine with Docker engine installed. In Azure you can provision an Ubuntu machine with Docker using “Docker on Ubuntu Server” as shown below

• Run the code by navigating to the directory where the repository was cloned and then run the following command

• Verify that the container is running by entering this command

• This should return the container that was just created in the previous step. The output should look something similar to below

• Finally push the docker image created (i.e. xyz/sampleapi) to the Docker image Registry

Orchestrating Microservices

• Service Fabric
• Docker Swarm
• DC/OS
• Kubernetes

Creating Kubernetes in Azure Container Service (ACS)

• Install Azure CLI using this link
• Login to Azure using Azure CLI by entering the following command

• Follow the steps to login to Azure using Azure CLI. Also if you have access to multiple subscriptions, make sure the right subscription is selected

• Create a resource group

• Create Kubernetes cluster in ACS under the resource group created

• Download the Kubernetes client by entering this command:

• Get the Kubernetes cluster credentials

• Tunnel into the Kubernetes cluster

• Open a web browser and navigate to https://localhost:8001/ui

Deploying the Web API

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: sampleapi-deployment
spec:
  replicas: 3
  template:
    metadata:
      labels:
        app: sampleapi
    spec:
      containers:
      - name: sampleapi
        image: xyz/sampleapi:latest
        ports:
        - containerPort: 80

• Deploy the app by opening a command line, navigating to where the yml file is located and running

• To expose the API outside of the cluster, run the following command

• Periodically check whether the service is exposed. Keep running this command until the service gets an External-IP

• Once the service is exposed, obtain its external IP (i.e. xxx.xxx.xxx.xxx) and check whether you can access the api by using either Rest client (i.e. Postman) or simply opening a web browser and entering:

Conclusion

• The monolithic nature of the solution makes scaling the app not resource efficient
• Code base for the app tends to be large which makes it harder to change and maintain
• Collaborating on monolithic apps is challenging since multiple teams could be making changes to the code simultaneously which increases the likelihood of merge conflicts
• A fatal error in one of the components could bring the entire solution down
• Teams are bound to a specific technology stack for the entire solution

• Ability to deploy subset of the system. If a customer is not interested in one of the modules, they can choose not to deploy that module
• Ability to scale specific components of the system. If “Feature1” is experiencing an unexpected rise in traffic, that component could be scaled without the need to scale the other components which enhances efficiency of infrastructure use
• Improves fault isolation/resiliency: System can remain functional despite the failure of certain modules
• Easy to maintain: teams can work on system components independent of one another which enhances agility and accelerates value delivery
• Flexibility to use different technology stack for each component if desired
• Enhances the ability of developers to gain domain knowledge for a specific area
• Allows for container based deployment which optimizes components’ use of computing resources and streamlines deployment process

1. Create a Team Project in VSTS called MySalesforceApp
2. Connect to MySalesforceApp from Eclipse IDE and clone the repository
3. Create a Force.com project, connect it to the Salesforce development instance and retrieve its metadata
4. Store the retrieved metadata in VSTS git version control system
5. Set up a Continuous Integration build and deploy to the Salesforce QA instance
6. Verify the CI behaves as expected

<project name="Sample usage of Salesforce Ant tasks" default="deployCodeAndRunTests" basedir="." xmlns:sf="antlib:com.salesforce">

    <property file="build.properties"/>
    <property environment="env"/>

    <!-- Setting default value for username, password and session id properties to empty string 
         so unset values are treated as empty. Without this, ant expressions such as ${sf.username}
         will be treated literally.
    -->
    <condition property="sf.username" value=""> <not> <isset property="sf.username"/> </not> </condition>
    <condition property="sf.password" value=""> <not> <isset property="sf.password"/> </not> </condition>
    <condition property="sf.sessionId" value=""> <not> <isset property="sf.sessionId"/> </not> </condition>

    <taskdef resource="com/salesforce/antlib.xml" uri="antlib:com.salesforce">
        <classpath>
            <pathelement location="ant-salesforce.jar" />        	
        </classpath>
    </taskdef>
			

    <!-- Deploy code and run tests.  If test fails, rollback deploy. -->
    <target name="deployCodeAndRunTests">
      <sf:deploy username="${sf.username}" password="${sf.password}" sessionId="${sf.sessionId}" serverurl="${sf.serverurl}" maxPoll="${sf.maxPoll}" deployRoot="..\src" testLevel="RunAllTestsInOrg" rollbackOnError="true" logType="Debugonly"/>
    </target>
		
</project>

# build.properties
#

# Specify the login credentials for the desired Salesforce organization
sf.username = yourUser
sf.password = YouPasswordAndSecurityTokenWithNotingInBetween
#sf.sessionId = 
#sf.pkgName = 
#sf.zipFile = 
#sf.metadataType = 

# Use 'https://login.salesforce.com' for production or developer edition (the default if not specified).
# Use 'https://test.salesforce.com for sandbox.
sf.serverurl = https://login.salesforce.com

sf.maxPoll = 20
# If your network requires an HTTP proxy, see https://ant.apache.org/manual/proxy.html for configuration.
#

public with sharing class Math {
    public Math(){
    
    }
    
    public Integer add(Integer a, Integer b){
        return a+b;
    }
}

@isTest 
private class MathTest {

    static testMethod void addTest() {
       Math m = new Math();
       Integer result = m.add(4, 5);
       System.assertEquals(result, 9);
    }

}

@isTest 
private class MathTest {

    static testMethod void addTest() {
       Math m = new Math();
       Integer result = m.add(4, 5);
       System.assertEquals(result, 10);
    }

}