AzureCAT Guidance https://docs.microsoft.com/archive/blogs/azurecat/feed.xml Hands-on solutions, with our heads in the Cloud! Fri, 07 Jun 2019 02:28:35 GMT en-US hourly 1 AzureCAT Blog - Moving to a new home https://docs.microsoft.com/archive/blogs/azurecat/azurecat-blog-moving-to-a-new-home Wed, 27 Feb 2019 12:51:45 GMT https://blogs.msdn.microsoft.com/azurecat/?p=5385
  • http://aka.ms/CAT
    • This is our last blog post on this site. So please head over to our new blog! The MSDN blogs are being removed. We'll miss this blog, and like all new homes, it will take some time getting used to our new location, but we're pretty excited about plugging into a vibrant new community! All the blog posts are migrated over there, so we're sorry if it breaks the old MSDN links (there are no plans for MSDN URLs to continue). So you might want to check the old URLs if you dropped links anywhere, and update them to the new blog (I can help a little with that). Thank you for reading our blog, and we look forward to our new future over in the Microsoft Tech Community:   AzureCAT Guidance "Hands-on solutions, with our heads in the Cloud!"]]>         Running Micro Focus Enterprise Server 4.0 in a Docker Container in Azure https://docs.microsoft.com/archive/blogs/azurecat/running-micro-focus-enterprise-server-4-0-in-a-docker-container-in-azure Wed, 27 Feb 2019 10:13:30 GMT https://blogs.msdn.microsoft.com/azurecat/?p=5025 Micro Focus Enterprise Server 4.0 and Enterprise Developer 4.0 in Azure.  There was a follow up to that blog outlining how to configure and deploy the BankDemo application that comes with Enterprise Developer.  I'd like to continue in this Micro Focus vein by outlining how to run Micro Focus Enterprise Server in a Docker Container. New to Micro Focus Enterprise Server 4.0 is the ability to run in a Docker container.  The benefits of running Enterprise Server in Docker are centered around portability, performance, agility and isolation.   For instance, a Docker image can be exported from one Windows VM to another or from another repository, and imported into another Windows server (assuming Docker support is installed there).  It can then be run in the same way without having to install Enterprise Server because it is a part of the image (licensing considerations do still apply).  For the purposes of this blog, I have decided to show how to set this up using the Windows 2016 Datacenter with Containers VM which is available from the Azure Marketplace. The nice thing about this offering is that Docker 18.09.0 is already preinstalled and ready to go.  All we will need to do is deploy the container, run it and then connect to it with a 3270 emulator.  At the end of this post, you will effectively be running a Mainframe application in a container. Let's get started. First a note about the media and licensing In order to complete this walk through, you will need to have access to the Micro Focus media.  This is available for Micro Focus customers and potential customers via a trial.  If you have access to Enterprise Server 4.0 or Enterprise Developer 4.0, you should have access to the accompanying Docker Support files.  If you do not have access, you can go here to get the process started. There are two sets of demonstrations available: one set for Enterprise Server and another for Enterprise Developer.  Within these two sets, there are demonstrations for Windows based platforms and SUSE Linux. For this blog you are going to walk through the Windows CICS acctdemo demonstration for Enterprise Server. The accompanying files are in a zip file named: ent_server_dockerfiles_4.0_windows.zip.  There is an accompanying licensing file with a name similiar to ES-Docker-Prod-XXXXXXXX.mflic that will be necessary in order to build the Docker images.  Once you have secured the media and the appropriate licensing file, you can begin.  
    1. Create the VM - The first thing we will need to do is create the VM in Azure via the Azure Portal.  As I mentioned before, we are going to use the Windows Server 2016 Datacenter - with Container VM which is available in the Azure Marketplace.  If you select Create Resource from the top left of the Portal and then filter by "Docker" and then select Windows Server 2016, you can select the VM.
     

    Next, you will need to configure the properties for the VM.  The Windows Server with Containers VM is only available in certain VM sizes.  I chose a Standard DS2_v2 with 2 vcpus and 7GB of memory.  You will also need to select the Region and Resource Group you would like to deploy to.  We do not need any Availability options for this walk through.  You will need to input the Username you would like to use for the Administrator Account as well as the Password.  Finally make sure port 3389 RDP is open.  This is the only port you need to publicly expose.  You will need it to log into the VM.  After this you can accept all of the default values and click Review+ Create.

       

    The deployment could take a couple of minutes.  When it is complete you will be presented with a message stating your VM has been created with a link to Go to Resource.  Click this and you will be taken to the Overview blade for your VM.  In the right pane, you will see a Connect button.  Click this and you will see the Connect to virtual machine dialog pop up on the right.  Click on the Download RDP File button.  This will download the RDP file that will allow you to attach to the VM.  After the file has finished downloading, open it and type in the Username and Password you created for the VM.  Make sure you are not using your corporate credentials to log on since the RDP client will assume you may want to use these.  You do not.  Select More Choices and you will be able to select your VM credentials.  After this, you will be logged into the VM.

    2. Create Sandbox Directory and Upload the zip File - Now that we have the machine running and attached via RDP, create a Sandbox Directory and upload the ent_server_dockerfiles_4.0_windows.zip and the ES-Docker-Prod-XXXXXXXX.mflic to that directory.  Extract the zip file contents to the ent_server_dockerfiles_4.0_windows directory (the extract will create this for you).  Inside this directory, you will have a README html and txt file (two versions of the same thing).  You will also have 2 directories EnterpriseServer and Examples.

    VERY IMPORTANT - Make sure you copy the licensing file ES-Docker-Prod-XXXXXXXX.mflic to the C:\Sandbox\ent_server_dockerfiles_4.0_windows\EnterpriseServer directory and the C:\Sandbox\ent_server_dockerfiles_4.0_windows\Examples\CICS.  As I mentioned before, these are necessary for the Docker build step to make sure the images are properly licensed.

    3. Check Docker Version and Create Base Image - Very Important: Creating the appropriate Docker image you need to use is a 2 step process.  First you will need to create the Enterprise Server 4.0 Base image.  After this is complete you will create another image for the x64 platform.  You also have the option to create a x86 (32 bit) image, but since you are running this in Azure, you will need the 64 bit image.  In order to do this, open a Command Prompt.

    • Check that Docker is installed.  At the command prompt, type docker version.  You should see a return similar to the image below.  At the time of  this post, the version is 18.09.0.  This lets us know that Docker is installed.
    • Next, we will need to switch to the C:\Sandbox\ent_server_dockerfiles_4.0_windows\EnterpriseServer. 
      • Type cd\Sandbox\ent_server_dockerfiles_4.0_windows\EnterpriseServer
      • Next type bld.bat IacceptEULA at the prompt.  This will kick off the build process for the initial base image.  This will take a few minutes to run.  When it is done, you will see a return like the following.  Notice the two images that have been created (one for x64 and x86).
     
    • Now we need to switch to the CICS example director and create the final image for the CICS demonstration.
      • Type cd\Sandbox\ent_server_dockerfiles_4.0_windows\Examples\CICS to switch to the CICS directory.
      • Now you need to create the image.  Type bld.bat x64.  Again, this will run for a few minutes.  When it is done, you should see a message stating that the image was created.
      • To confirm this, type docker images.  You will see a list of all of the Docker images installed on the VM. Make sure microfocus/es-acctdemo is one of them.
    • Now lets run the image.  This will launch Enterprise Server 4.0 and the acctdemo application.
      • At the command prompt type docker run -p 16002:86/tcp -p 16002:86/udp -p 9040-9050:9040-9050 -p 9000-9010:9000-9010 -ti --network="nat" --rm microfocus/es-acctdemo:win_4.0_x64.  Now the image is running and we can can attach to it via a 3270 emulator via port 9040.  For this walk though I used x3270, but you can use whichever one you prefer.  Go ahead and install the emulator.
      • Next we will need to get the ip address of the acctdemo container.  Docker acts as a DHCP Server for the Containers is manages.
        • First, we need to get the ID of the running Container.  Type Docker ps at the command prompt and note the ID.  You will need this to get the IP address for the acctdemo Container.  In this example it is 22a0fe3159d0.
        • To get the ip address type docker inspect 22a0fe3159d0 --format="{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}".  You should be presented with the ip address for the acctdemo image like in the image below.
    • Now let's mount the image using the emulator.  Configure the emulator to use the address of the acctdemo image and port 9040.  For this walk though it would be 172.19.202.52:9040.  Yours's will be similiar. You should be presented by a CICS logon screen like the one below.
    • Now you need to logon to the CICS Region.  You can do this by entering SYSAD for the USERID and SYSAD for the Password.
    • Next you need to clear the screen.  You can find this on the keymap of the emulator you are using.  For x3270 select the Keymap menu option.
    • To launch the acctdemo application type ACCT.  You should see the initial screen for the application.
     
    • If you type D for the Request and 11111 for the ACCOUNT, you be taken to some fictional account information.  Other account numbers to try are 22222, 33333....
     
    • Finally, to administer the Enterprise Server, type start http:172.19.202.52:86 from a command prompt.  This will bring up the Enterprise Server Administration console.
      Summary Now you are running and managing a CICS application in a Docker Container.  Hopefully this was interesting.  In the future we will explore leveraging Kubernetes in Azure (AKS) for orchestrating and managing the containers.]]>         2018 Guidance from AzureCAT: SAP on the Microsoft Platform https://docs.microsoft.com/archive/blogs/azurecat/2018-guidance-from-azurecat-sap-on-the-microsoft-platform Wed, 20 Feb 2019 09:25:15 GMT https://blogs.msdn.microsoft.com/azurecat/?p=5015 (refresh)". Bold articles are showcased in more detail at the bottom of this page.  

    Showcase

    SAP S/4HANA for Linux Virtual Machines on Azure This reference Architecture by Ben Trinh, Ross Sponholtz, and Amish Patel shows a set of proven practices for running S/4HANA in a high availability environment that supports disaster recovery on Azure. This architecture is deployed with specific virtual machine (VM) sizes that can be changed to accommodate your organization's needs.   Run SAP HANA on Azure Large Instances This reference architecture by Ben Trinh, Ross Sponholtz, and Amish Patel shows a set of proven practices for running SAP HANA on Azure (Large Instances) with high availability and disaster recovery (DR). Called HANA Large Instances, this offering is deployed on physical servers in Azure regions.   SAP NetWeaver (Windows) for AnyDB on Azure VMs This reference architecture by Ben Trinh, Ross Sponholtz, and Amish Patel shows a set of proven practices for running SAP NetWeaver in a Windows environment on Azure with high availability. The database is AnyDB, the SAP term for any supported DBMS besides SAP HANA. This architecture is deployed with specific virtual machine (VM) sizes that can be changed to accommodate your organization's needs.   SQL files and backups on Azure for SAP As more users migrate their SAP workloads to Azure, they will discover the benefits of migrating the Microsoft SQL Server data tier as well. Storing SQL Server data files on Azure can help speed migration, improve security, and provide high availability and disaster recovery. This guide by Ravi Alwani of AzureCAT addresses common questions from SAP users. It explains how to store SQL Server data files using Azure Blob Storage and set up file-snapshot backups, which provide nearly instantaneous backups and quicker restores for database files stored using the Azure Blob Storage service.       AzureCAT Guidance "Hands-on solutions, with our heads in the Cloud!"  ]]>         NEW REFERENCE ARCHITECTURE: Batch scoring of Spark models on Azure Databricks https://docs.microsoft.com/archive/blogs/azurecat/new-reference-architecture-batch-scoring-of-spark-models-on-azure-databricks Mon, 18 Feb 2019 09:20:33 GMT https://blogs.msdn.microsoft.com/azurecat/?p=5365 AI reference architecture (on the Azure Architecture Center) is written by AzureCAT John Ehrlinger, and published by Mike Wasson. Reference architectures provide a consistent approach and best practices for a given solution. Each architecture includes recommended practices, along with considerations for scalability, availability, manageability, security, and more. The full array of reference architectures is available on the Azure Architecture Center. Batch scoring of Spark models on Azure Databricks   This reference architecture shows how to build a scalable solution for batch scoring an Apache Spark classification model on a schedule using Azure Databricks, an Apache Spark-based analytics platform optimized for Azure. The solution can be used as a template that can be generalized to other scenarios. The architecture defines a data flow that is entirely contained within Azure Databricks based on a set of sequentially executed notebooks. It consists of the following components:
    • Data files. The reference implementation uses a simulated data set contained in five static data files.
    • Ingestion. The data ingestion notebook downloads the input data files into a collection of Databricks data sets. In a real-world scenario, data from IoT devices would stream onto Databricks-accessible storage such as Azure SQL Server or Azure Blob storage. Databricks supports multiple data sources.
    • Training pipeline. This notebook executes the feature engineering notebook to create an analysis data set from the ingested data. It then executes a model building notebook that trains the machine learning model using the Apache Spark MLlib scalable machine learning library.
    • Scoring pipeline. This notebook executes the feature engineering notebook to create scoring data set from the ingested data and executes the scoring notebook. The scoring notebook uses the trained Spark MLlib model to generate predictions for the observations in the scoring data set. The predictions are stored in the results store, a new data set on the Databricks data store.
    • Scheduler. A scheduled Databricks job handles batch scoring with the Spark model. The job executes the scoring pipeline notebook, passing variable arguments through notebook parameters to specify the details for constructing the scoring data set and where to store the results data set.
      Topics covered include:
    • Architecture
    • Recommendations
    • Performance considerations
    • Storage considerations
    • Cost considerations
    • Deploy the solution
    • Related architectures

    See also

    Additional related AI reference architectures:   Find all our reference architectures here.   AzureCAT Guidance "Hands-on solutions, with our heads in the Cloud!"]]>         NEW: Mainframe Migration on the Azure Cloud Adoption Framework https://docs.microsoft.com/archive/blogs/azurecat/new-mainframe-migration-on-the-azure-cloud-adoption-framework Fri, 15 Feb 2019 10:01:17 GMT https://blogs.msdn.microsoft.com/azurecat/?p=5295 Cloud Adoption Framework (previously Fusion and Enterprise Cloud Adoption) launched on Monday, February 11! The cloud presents a fundamental shift in the way enterprises procure and utilize technology resources. In the past, enterprises assumed ownership and responsibility of all levels of technology from infrastructure to software. Now, the cloud offers the potential to transform the way enterprises utilize technology by provisioning and consuming resources as needed. The Cloud Adoption Framework (CAF) provides the desired framework to guide decisions throughout adoption. As part of this launch, we're showcasing four new articles from Larry Mead.   Mainframe migration overview Mainframe migration overview Many companies benefit from moving some or all their mainframe workloads, applications, and databases to the cloud. Azure provides mainframe-like features at cloud scale without many of the drawbacks associated with mainframes. Migrating to the cloud enables companies to modernize their infrastructure. With cloud services you can make mainframe applications, and the value that they provide, available as a workload whenever your organization needs it. Most Fortune 500 companies are already running Azure for their critical workloads. Azure's significant bottom-line incentives motivate many migration projects.   Mainframe myths and facts Mainframe myths and facts Mainframes figure prominently in the history of computing and remain viable for highly specific workloads. Most agree that mainframes are a proven platform with long-established operating procedures that make them reliable, robust environments. Software runs based on usage, measured in million instructions per second (MIPS), and extensive usage reports are available for charge backs. The reliability, availability, and processing power of mainframes have taken on almost mythical proportions. To evaluate the mainframe workloads that are most suitable for Azure, you first want to distinguish the myths from the reality.   Make the switch from mainframes to Azure Make the switch from mainframes to Azure (migration strategies) As an alternative platform for running traditional mainframe applications, Azure offers hyperscale compute and storage in a high availability environment. You get the value and agility of a modern, cloud-based platform without the costs associated with a mainframe environment. This article provides technical guidance for making the switch from a mainframe platform to Azure.   Mainframe application migration Mainframe application migration (application strategies) When migrating applications from mainframe environments to Azure, most teams follow a pragmatic approach: reuse wherever and whenever possible, and then start a phased deployment where applications are rewritten or replaced. Application migration typically involves one or more of the following strategies: rehost, retire, rebuild, and replace. You should consider starting by planning those workloads that you want to initially migrate, and then determine those requirements for moving associated applications, legacy codebases, and databases.     AzureCAT Guidance "Hands-on solutions, with our heads in the Cloud!"]]> CATwalk Author Bio: Larry Mead https://docs.microsoft.com/archive/blogs/azurecat/catwalk-author-bio-larry-mead Wed, 13 Feb 2019 09:13:50 GMT https://blogs.msdn.microsoft.com/azurecat/?p=4985 https://techcommunity.microsoft.com/t5/AzureCAT/CATwalk-Author-Bio-Larry-Mead/ba-p/333969 ======================== And just like that, we're on our third CATwalk Author Bio! CATwalk is our new series of blogs where you get to meet the CATs who make our content possible. Let me introduce you to Larry Mead. Larry Mead Larry is an AzureCAT Program Manager with more than 30 years of experience in information technology with solution sales, enterprise application architecture, system design, and development of highly available, highly scalable mission critical systems for enterprise clients in a heterogeneous environment. His work experience includes solution selling for large enterprise clients, incubation of new business groups, international business, complex partner enablement and management, and solution architecture and design. Larry enjoys helping customers succeed with data and analytics solutions. Larry has worked at Microsoft for almost 24 years. Prior to Microsoft, he worked as a Development Director at BancTec and a Development Manager at Continental Federal Savings Bank. Larry's technology experience areas include:
    • Azure (IaaS, PaaS)
    • AWS (IaaS)
    • Containers – Docker and Kubernetes
    • Relational databases (SQL Server, Oracle, Sybase, DB2, Informix)
    • Big Data (Hadoop and NoSQL)
    • Data warehouse and business intelligence
    • Predictive analytics and machine learning
    • Microsoft Windows (Server, Client, Microsoft Azure)
    • Mainframes (IBM, Unisys)
    • UNIX (Solaris, AIX, HP-UX, SCO)
    • Linux (Red Hat, SUSE)
    • Virtualization (Hyper-V, VMware)
    • Integration technologies
    • Development languages (Java, C#, C/C++, Visual Basic, COBOL, Assembler)
      Educational degrees vary widely among folks at Microsoft, and Larry is no exception, holding a BA in psychology from George Mason University.   Microsoft Azure Government cloud for mainframe applications Larry has already published an article this year, Microsoft Azure Government cloud for mainframe applications. This quick guide shows how United States government agencies and their partners can use Azure Government for mainframe applications. Azure Government delivers the advantages of a mainframe in a more cost-efficient and agile environment. Read the article. Just this week, we published four articles from Larry on the Azure Architecture Center's new Cloud Adoption Framework:
    1. Mainframe migration overview
    2. Mainframe myths and facts
    3. Make the switch from mainframes to Azure
    4. Mainframe application migration
                           In 2018, Larry contributed a number of articles on mainframe migration, including Deploy IBM Db2 pureScale on Azure and Demystifying mainframe to Azure migration. Both articles are featured in the Mainframe Migration to Azure - Portal. You can find all of Larry's content using his tag:   AzureCAT wants to thank Larry for his contributions to the Microsoft community thus far, and we look forward to much more!   AzureCAT Guidance "Hands-on solutions, with our heads in the Cloud!"]]>         Service Fabric Customer Architecture: ZEISS Group https://docs.microsoft.com/archive/blogs/azurecat/service-fabric-customer-architecture-zeiss-group Mon, 11 Feb 2019 09:41:01 GMT https://blogs.msdn.microsoft.com/azurecat/?p=5075 https://techcommunity.microsoft.com/t5/AzureCAT/Service-Fabric-Customer-Architecture-Zeiss-Group/ba-p/333958 ======================== Visit Microsoft Customer Stories to see this article in a case study format. ZEISS

    The specialists who develop and manufacture the digital instruments at ZEISS Group have plenty on their minds. The sophisticated instruments they build need to work reliably and stay up to date. To serve business customers and end users better, the ZEISS Group built an Azure-based digital integration platform with Azure Service Fabric at its center.

    Some of our devices such as the robotics instruments used in microsurgery are highly complex. We didn't want to give the engineers developing these devices and other customer touch points the added headache of integration. Service Fabric was a key piece in making our digital integration platform as simple as possible. —Kai Walter, Lead Solution Architect at Carl Zeiss AG

    Headed by Carl Zeiss AG, a company with a venerable history that includes the invention of the compound microscope in 1857, the ZEISS Group performs extensive research and development in optics and photonics. As pioneers in scientific optics, ZEISS develops, produces, and distributes measuring technology, medical technology, eyeglass lenses, camera and cinema lenses, binoculars, and semiconductor manufacturing equipment. To streamline the customer experience, a development team at the ZEISS Group embarked on a digital integration project. The goal was to build an interoperability platform that allows field devices to connect to back-end business systems from SAP, Salesforce, and others, where customer records, device data, service history, and other information was stored. Customers would then have access to the latest software updates and support information. The solution needed to be simple. The ZEISS team did not want to add to the complexity faced by the users of their highly advanced devices. Yet the small development team wanted to work quickly, so they sought help from PlanB. Gmbh, a Microsoft Gold partner that specializes in cloud infrastructures and Azure implementations. PlanB. helped the team to create an Enterprise Service Bus in the cloud, capable of integrating the various devices and customer touch points via standard REST-based APIs. Reusable APIs could also be used by other developers and partners worldwide to connect devices to the customer service, shopping, and knowledge assets stored in the back-office systems. For more information about the business drivers behind this platform and its user credential service, see the Zeiss business story.

    A layered Azure sandwich

    The digital integration platform was designed to connect devices and other customer touch points (the consumers) in the simplest way possible given the various back-end systems (the producers). The solution combines several Azure PaaS (platform as a service) offerings. If the Azure Marketplace had a useful service, the team was willing to try it. This experimental approach provided a working prototype quickly while minimizing the typical costs and time associated with spinning up traditional servers and services. In addition, the team was still learning about what Azure could do. Azure PaaS gave the team the perfect low-risk, low-cost laboratory to learn in. Through this process of explore-expand-extract, the ZEISS team discovered Service Fabric, the environment for building large-scale, highly available, distributed cloud applications. Service Fabric works like an orchestration engine for deploying microservices across a cluster of machines and provides developers with the tools they need to design apps and services that run at cloud scale. After a few weeks of continuous evolution, the developers came up with the initial design, which placed Azure services in the middle of a sandwich with APIs on top, and back-office systems on the bottom. At the top layer, consumers interact with the system through the APIs that are exposed through Azure API Management. This layer plays a key role in hiding the complexity of downstream transactions from consumers, and makes it easy for the ZEISS team to manage the API life cycle. As new consumers are added to the platform, the ZEISS development team works with them to create the APIs they need. With the capability to mock up responses based on the joint API agreement, this layer decouples the development teams, enabling consumer teams to begin working with the APIs immediately regardless of back-end development. The middle layer transports, transforms, and holds state for the information passing through the digital integration platform. For stateless scenarios, API Management passes incoming messages to Azure Functions, which performs the transformations needed to enable consumers and producers to communicate—for example, transforming from queues to REST APIs and vice versa. The development team chose Functions because of its simple programming model with declarative bindings and few lines of code. They could focus on writing the transformations they needed with less risk and grow with the innovations in the Azure Functions host environment. and allow other services to handle infrastructure complexity. However, Functions, when run in a consumption plan, does not support virtual network integration. So they created an Azure DevOps pipeline to package the function code together with the Functions runtime into containers and to handle deployment. The developers didn't have to change any of their code. Service Fabric handles their stateful services and provides a runtime environment that supports their virtual network. To hold state for long-running, multiple step processes, the developers used an actor-pattern-based application framework built on top of Service Fabric Reliable Services. Each Reliable Actor service is a partitioned, stateful Reliable Service. Information is then communicated to the appropriate back-end system, which matches it to the desired result and reports back via the API to Service Fabric, which communicates the result to the consumer. This layer also includes Azure Service Bus queues to support asynchronous messaging between services that require it. In addition, some information is persisted via Azure Cosmos DB, a multi-model database service on a globally distributed, managed platform. When Azure Cosmos DB is used in combination with the binding capabilities in Azure Functions, developers can store and retrieve data with much less code than is required for storing data in a traditional relational database. For more complex processes, a business process layer based on SAP services orchestrates data and processes for enterprise resource planning (ERP) in the downstream systems. The bottom layer includes the back-end producers such as the SAP and Salesforce enterprise systems.

    Azure allowed us to grow. We started with public services then moved into private virtual networks, step by step. —Kai Walter, Lead Solution Architect at Carl Zeiss AG

    ZEISS customer story

    Evolution of the digital integration platform

    At first, the developers used Service Fabric only for the use cases where they needed to store state and retain it for the duration of a transaction. Service Fabric makes state highly available without the need to persist state to an external store. As the team became more familiar with Service Fabric, they began relying on it in more use cases. Stateless services were written as small pieces of code in Azure Functions—another key PaaS component. Azure Functions is a serverless compute service that enables developers to run code on demand without having to explicitly provision or manage infrastructure. The ZEISS developers wanted to spend their time perfecting the services of their integration platform, not thinking about the infrastructure. With this in mind, they are evaluating Service Fabric Mesh, a fully managed service that frees developers from cluster management. Azure Functions ran in a consumption plan that automatically allocated compute power when the code was running. It scaled out as necessary to handle the load, and then scaled down when the code was not running. However, the team wanted more control over scaling. After six months, they switched to self-hosting of the Azure Functions runtime—a move that also gave Azure Functions the ability to directly access the back-end services running in the ZEISS datacenter. As a security precaution, transactions involving certain types of data do not leave the corporate network. When the team set up Service Fabric, they also created an ExpressRoute connection from the Azure virtual network to the ZEISS datacenter. This connection enabled Service Fabric to operate as if it was running in the datacenter. Further, it gave the developers access to datacenter resources and added control over where transactions occur, such as those handled by Service Bus and Azure Cosmos DB. The developers wanted to give Azure Functions greater access to the back-end systems but keep the lambda notation and declarative development paradigm of Functions. They realized they could host Functions in containers within Service Fabric, which was already connected to the ZEISS datacenter. This move made Service Fabric central to all integration tasks, not just the stateful transactions.

    Services in the digital integration platform

    The success of the integration platform was driven in large part by three critical stateful services that run in Service Fabric:
    • Software Update service. This stateful service processes customer requests to upgrade their software, a feature that ZEISS offers on demand for their digital instruments. This service triggers and waits for order and license processing on several back-end systems, issues a license file directly to the requesting customer, and collects and preserves the status for better traceability.
    • ZEISS Global Mail API service. The platform enables cloud-based consumers to send authentic “ZEISS” email over the corporate IT email back-end. These consumers include web shops and devices that would not otherwise have direct access to the company’s back-end email service. This stateful service waits for completion of the email sending process and can notify the original requestor upon success or failure. Designed as a simple-to-use entry-level service, it also entices more developers to sign up for and use the integration platform APIs in their work. From the ZEISS developer portal, users can immediately sample the service, send a test email message using the programming language of their choice, and see for themselves how easy the APIs in general are to use.
    • Compliance Check service. This stateful service runs a compliance check for each transaction involving delivery of goods that may be subject to export restrictions. The service verifies whether the goods can be delivered to a particular address, country, or company.

    Our intention was to start fast and start small, and so we used Azure PaaS first. When we hit certain limitations of PaaS, we discovered Service Fabric. It became the centerpiece of our more sophisticated scenarios. —Kai Walter, Lead Solution Architect at Carl Zeiss AG

    Many Azure services make a sandwich

    ZEISS is heavily invested in SAP enterprise systems, and the digital integration platform needed to integrate with several, including SAP HCM, SAP ECC, and SAP CRM. ZEISS passes about a half-million SAP messages a day through the platform. As a result, balancing load proved to be a critical feature, because the enterprise systems on the back end could not operate at cloud speed, and some of the consumers couldn’t handle the number of messages sent by SAP either. The platform adjusts automatically to the load by using different messaging patterns to enable communication across the layers:
    • Synchronous: A consumer can send a request through API Management, and the platform routes it to producer, performing basic transformations and applying authentication or token validation as needed.
    • Asynchronous in: When producers cannot handle the rate of incoming traffic, a request is placed in a Service Bus queue. A function reads the queue and communicates at a pace the producer can accept, sending only one message at a time from the queue.
    • Asynchronous out: A producer can send a message via API through a function back to a queue for asynchronous processing. The function then pushes it to the consumer.
    • Decoupled: Some consumers cannot accommodate the number of documents sent by the SAP producers. In these cases, the documents are buffered in Azure Cosmos DB until the consumer pulls them using an API call.
    To keep upstream and downstream traffic running smoothly, Service Fabric is at the center of all transactions. When an SAP transaction occurs, the data must be unloaded as fast as possible to avoid placing undue load on the servers. The upstream flow of data must be handled in the most performant way possible. So, the ZEISS developers used the sandwich of Azure components to serve as a buffer. Users and devices communicate through the API, where Azure API Management throttles and measures the API calls to the platform. Other Azure components are used to throttle downstream. On the transport layer, Azure Cosmos DB is used to persist state or buffer documents passing through the platform from some SAP producers back to consumers. In addition, the digital integration platform used a few other Azure services:
    • Azure Active Directory B2C is the identity management service used to authenticate ZEISS customers on apps, portals, and shops. It works in conjunction with token validation performed on API call-level safeguards to provide authorization to use certain back-end capabilities.
    • Active Directory Federation Services is the identity management service used to authenticate ZEISS employees who log on to the developer portal for the digital integration platform.
    • Azure Key Vault stores cryptographic keys and secrets associated with various platform services.
    • Azure Application Insights enables the team to monitor the platform, diagnose issues, and keep track of API usage and life cycle.
    • Azure DevOps and the entire Azure service ecosystem support efficient development based on an infrastructure-as-code approach using scripts and templates.

    Key advantage in Service Fabric

    Service Fabric is the center of the ZEISS digital integration platform and offered the development team several key benefits:
    • One compute platform. Service Fabric runs all transactions at the scale they require.
    • Reliable Actors. An application framework based on the Reliable Actor service holds state in a consumer context, which was necessary for transactions that take place across several steps in the Compliance Check and License Update services.
    • App management. The developers integrated Service Fabric with the continuous integration and continuous deployment (CI/CD) capabilities of Azure DevOps. Service Fabric provides support for the full application life cycle and CI/CD of cloud applications.
    • A platform that grows with the team. The ZEISS development team turned to Service Fabric use case by use case. Within six months, using an evolutionary approach with little capital binding and low capital risk, the team was able to deliver a fully fledged integration platform.
    • Deploy—and redeploy. The team learned by trial, measuring and improving each iteration and making the most of Azure PaaS and the ability to turn services on and off. With Service Fabric, the developers can delete deployed services as easily as they can provision, deploy, monitor, and patch them.
    • Cost-effective performance. In moving Azure Functions to Service Fabric containers, the team got an added benefit. They pay once to process both stateless functions and stateful Service Fabric programming models.

    Conclusion

    Azure services, and Service Fabric in particular, are at the center of a digital transformation at ZEISS. The digital integration platform supports a visionary effort to make smart devices and prepare ZEISS for the future. With secure access to ZEISS enterprise systems, consumers in the digital integration platform can schedule their own service calls and give ZEISS designers predictive insight into customer needs. With the digital integration platform in place, now the team at ZEISS is investing in developing standard patterns and building integrations in a factory approach.

    Service Fabric was the key because it allowed us to run all our stateful and stateless interactions on one compute platform that we can scale on our terms. Everything is in Service Fabric. —Kai Walter, Lead Solution Architect at Carl Zeiss AG

      Visit Microsoft Customer Stories to see this article in a case study format.   AzureCAT Guidance "Hands-on solutions, with our heads in the Cloud!"]]>         NEW EXAMPLE SCENARIO: Accelerate digital image-based modeling on Azure https://docs.microsoft.com/archive/blogs/azurecat/new-example-scenario-accelerate-digital-image-based-modeling-on-azure Fri, 08 Feb 2019 09:03:00 GMT https://blogs.msdn.microsoft.com/azurecat/?p=5055 https://techcommunity.microsoft.com/t5/AzureCAT/NEW-EXAMPLE-SCENARIO-Accelerate-digital-image-based-modeling-on/ba-p/333956 ======================== Written by Paulo Marques da Costa. Edited by Nanette Ray. Published by Adam Boeglin from Microsoft patterns & practices. This example scenario provides architecture and design guidance for any organization that wants to perform image-based modeling on Azure infrastructure-as-a-service (IaaS). The scenario is designed for running photogrammetry software on Azure Virtual Machines using high-performance storage that accelerates processing time. The environment can be scaled up and down as needed and supports terabytes of storage without sacrificing performance. Accelerate digital image-based modeling on Azure The concepts in this article apply to any high-performance computing (HPC) workload based on a scheduler and worker nodes managed as infrastructure. For this workload, Avere vFXT was selected for its superior performance during benchmark tests. However, the scenario decouples the storage from the processing so that other storage solutions can be used. Components included in this example scenario:   Topics covered include:
    • Relevant use cases
    • Architecture
      • Components
      • Alternatives
    • Considerations
      • Security
    • Pricing
    • Deployment
    • Related Resources
      For step-by-step instructions for deploying this architecture, including all the prerequisites for using either Avere FxT or BeeGFS, download the ebook: Deploy Agisoft PhotoScan on Azure With Avere vFXT for Azure or BeeGFS.   AzureCAT Guidance "Hands-on solutions, with our heads in the Cloud!"]]>         CATwalk Author Bio: Rakesh Patil https://docs.microsoft.com/archive/blogs/azurecat/catwalk-author-bio-rakesh-patil Thu, 07 Feb 2019 09:15:28 GMT https://blogs.msdn.microsoft.com/azurecat/?p=4835 https://techcommunity.microsoft.com/t5/AzureCAT/CATwalk-Author-Bio-Rakesh-Patil/ba-p/333951 ======================== Welcome back to our CATwalk Author Bio series! Just a refresher, we're the AzureCAT team. And since "CAT" stands for Customer Advisory Team, saying "AzureCAT team" is like saying "We're the Azure Customer Advisory Team team." … which is weird. So just call us AzureCAT. Nomenclature aside, CATwalk is our new series of blogs where you get to meet the CATs who make our content possible. For our second CATwalk, you get to pal around with Rakesh Patil.   Rakesh is an AzureCAT Senior Program Manager from India. He has a range of experience and is well-versed in the Linux and UNIX operating systems, storage, micro services, and modern workloads. Rakesh is currently working with the leading Linux operating system vendors and providing solutions to enterprise customers. Rakesh graduated from Sikkim Manipal University of Health, Medical, and Technology  (in India) with a BCA degree, Bachelor of Computer Applications. Rakesh joined the tech industry in 2004. He worked as a systems associate for Tech Mahindra and HCL before joining Orange as a senior systems administrator. Rakesh later moved to Red Hat and held the same position. He is a Red Hat Certified Engineer, and NetApp and Virtualization Administrator. In 2016, Microsoft was fortunate enough to have Rakesh join AzureCAT, where he continues to share his knowledge with our customers.    Parallel Virtual File Systems on Microsoft Azure Rakesh is an active AzureCAT contributor and has authored or co-authored a number papers and e-books. In April of 2019, Rakesh co-authored the Parallel Virtual File Systems on Microsoft Azure e-book, which documents the results of a series of performance tests on Azure to see how scalable Lustre, GlusterFS, and BeeGFS are. You can download the e-book or read it in installments.   In September 2017, Rakesh co-authored Azure GAHP Server for HTCondor, an article about the collaboration between AzureCAT and the University of Wisconsin resulting in the development of an Azure GAHP (Grid ASCII Helper Protocol) specification and an Azure GAHP server. Together they enable the HTCondor framework to use Azure for running high-throughput computing (HTC) workloads using Azure Virtual Machines and virtual machine scale sets.   You can find a complete listing of Rakesh's work and guidance using his AzureCAT tag:   AzureCAT thanks Rakesh Patil for his contributions to the Microsoft community thus far, and we look forward to more great work!   AzureCAT Guidance "Hands-on solutions, with our heads in the Cloud!"    ]]> NEW REFERENCE ARCHITECTURE: Enterprise-grade conversational bot https://docs.microsoft.com/archive/blogs/azurecat/new-reference-architecture-enterprise-grade-conversational-bot Tue, 05 Feb 2019 09:04:43 GMT https://blogs.msdn.microsoft.com/azurecat/?p=4965 https://techcommunity.microsoft.com/t5/AzureCAT/NEW-REFERENCE-ARCHITECTURE-Enterprise-grade-conversational-bot/ba-p/333945 ======================== Our seventh AI reference architecture (on the Azure Architecture Center) is written by AzureCATs Abhinav Mithal and Robert Alexander, and published by Mike Wasson. Reference architectures provide a consistent approach and best practices for a given solution. Each architecture includes recommended practices, along with considerations for scalability, availability, manageability, security, and more. The full array of reference architectures is available on the Azure Architecture Center. Enterprise-grade conversational bot This reference architecture describes how to build an enterprise-grade conversational bot (chatbot) using the Azure Bot Framework. Each bot is different, but there are some common patterns, workflows, and technologies to be aware of. For a bot to serve enterprise workloads, there are many design considerations to ponder beyond the core functionality. This article covers the most essential design aspects, and introduces the tools needed to build a robust, secure, and actively learning bot. This reference architecture uses a significant number of Azure services. Your own bot may not use all of these services, or may incorporate additional services. Check out the article for additional detail on the services. Bot logic and user experience Bot cognition and intelligence Data ingestion Logging and monitoring Security and governance Quality assurance and enhancements   Topics covered include:
    • Architecture
      • Bot logic and user experience
      • Bot cognition and intelligence
      • Data ingestion
      • Logging and monitoring
      • Security and governance
      • Quality assurance and enhancements
    • Design considerations
      • User message flow
      • System data flow
    • Building a bot
      • Ingest data
      • Core bot logic and UX
      • Add smarts to your bot
    • Quality assurance and enhancements
    • Availability considerations
    • Security considerations
    • Manageability considerations
      • Monitoring and reporting
      • Automated resource deployment
      • Continuous bot deployment
    Head over to the Azure Architecture Center to learn more about the Enterprise-grade conversational bot reference architecture.

     

    See also

    Additional related AI reference architectures:   Find all our reference architectures here.   AzureCAT Guidance "Hands-on solutions, with our heads in the Cloud!"  ]]>