Deploy Devito on an Azure virtual machine

This article describes how to run Devito on an Azure virtual machine (VM). It also presents the performance results of running Devito on Azure.

Devito is a functional language that you can implement as a Python package. With Devito, you can use high-level symbolic problem definitions to create optimized stencil computation, such as finite differences, image processing, and machine learning. Devito is built on SymPy and uses automated code generation and just-in-time compilation to run optimized computational kernels on several compute platforms, including CPUs, GPUs, and clusters.

Architecture

The following diagram shows a single-node architecture:

Download a Visio file of this architecture.

The following diagram shows a multi-node architecture:

Download a Visio file of this architecture.

Components

• Azure CycleCloud is an enterprise-friendly tool that's used to orchestrate and manage HPC environments on Azure.

• Azure Virtual Machines is used to create a Linux VM. For information about how to deploy a VM and install drivers, see Linux VMs on Azure.

• Azure Virtual Network is used to create a private network infrastructure in the cloud.

• Network security groups are used to restrict access to the VM.

• A public IP address connects the internet to the VM.

• An Azure Blob Storage physical solid-state drive (SSD) is used for storage.

• The Azure CycleCloud REST API is used to add automated and programmatic cluster management capabilities, like determining a cluster's status or creating nodes.

Scenario details

Devito provides key offerings like:

• Mechanisms to adjust finite difference discretization.
• Constructs to express various operators.
• A flexible API.
• The ability to generate highly optimized parallel code.
• Distributed NumPy arrays.
• Smooth integration with popular Python packages.

Deploy Devito on Azure to get benefits like:

• Modern and diverse compute options to meet your workload's needs.
• The flexibility of virtualization without the need to buy and maintain physical hardware.
• Rapid provisioning.

Devito provides a functional language to implement sophisticated operators that can be made up of multiple stencil computations, boundary conditions, sparse operations (for example, interpolation), and more. With Devito, you might use explicit finite difference methods to approximate partial differential equations. For example, you can implement a 2D diffusion operator by using the following equation:

An operator generates low-level code from an ordered collection of Eq. This example is for a single equation.

There's virtually no limit to the complexity of an operator. The Devito compiler automatically analyzes the input, detects and applies optimizations (including single-node and multi-node parallelism), and generates code with suitable loops and expressions.

Install Devito

Before you install Devito, you need to deploy and connect a Linux VM, and install the required AMD and InfiniBand drivers.

For information about deploying the VM and installing the drivers, see Run a Linux VM on Azure.

After you deploy the Linux VM, see the Devito installation instructions to learn about three methods for installing Devito on your VM:

• Docker installation
• Pip installation
• Conda environment installation

Compute sizing and drivers

The Devito performance tests that are presented in the next sections used HBv3-series VMs running Linux. The following table provides details about these VMs:

Devito performance results

Benchmarking Devito on Azure

To test the performance of Devito on Azure, benchmarking was performed by using the HB120rs_v3 series SKU. There are many seismic models, like acoustic, tti, elastic, and visco-elastic, available on the tutorials page of the Devito website. The tests in this article use a forward operator under the acoustic model for benchmarking the performance of Devito.

The following table provides information about the operating system that was used for testing:

Benchmarking a Devito operator

You can use the benchmark.py python file to test the performance of a Devito operator. The file is located in the /benchmarks/user folder that's in the Devito folder. The benchmark.py file implements a minimalist framework to evaluate the performance of a Devito operator, while varying:

• The problem size, for example the shape of the computational grid.

• The discretization, for example the space-order and time-order of the input/output fields.

• The simulation time (in milliseconds).

• The performance optimization level.

• The autotuning level.

Devito performance on the HB120rs_v3 series (single-node)

The Devito forward operator performance for the acoustic model was tested on Standard HBv3 series virtual machines with 16, 32, 64, 96, and 120 vCPU configurations.

The following table shows the results for the CentOS-based 8.1 HPC image:

Note that for the single-node tests, the Devito operator is run on all HBv3-series VM configurations. The Standard_HB120-16rs_v3 VM runtime is used as the baseline to calculate the relative speed increase.

Devito performance on a cluster (multi-node)

The forward operator performance in the single-node tests show the scale-up behavior for the 64 and 96 vCPU configurations. The following performance tests run the Devito operator on two cluster configurations with 64 vCPUs and 96 vCPUs, respectively. The CentOS-based 8.1 HPC image is used for these two clusters.

The following table shows the results for a cluster with 64 vCPUs per node:

The following table shows the results for a cluster with 96 vCPUs per node:

Azure cost

The following table presents the wall-clock times for running the simulations. You can multiply these times by the Azure VM hourly costs for HB120rs_v3-series VMs to calculate costs. For the current hourly costs, see Linux virtual machines pricing.

The following runtimes represent only the simulation time. Application installation time isn't considered.

You can use the Azure pricing calculator to estimate the costs for your configuration.

The following table shows runtimes for the HB120rs_v3 series:

Summary

• Devito was successfully deployed and tested on the HB120rs_v3 series VM on Azure.

• For the single-node configuration, the Devito scales well up to 64 and 96 cores. It has a maximum scale up of 1.57 times with 64 cores.

• For the multi-node configuration, there's a gradual scale up from one node to eight nodes in both the clusters with the Standard_HB120-64rs_v3 and the Standard_HB120-96rs_v3 virtual machines.

Contributors

This article is maintained by Microsoft. It was originally written by the following contributors.

Principal authors:

• Hari Bagudu | Senior Manager
• Gauhar Junnarkar | Principal Program Manager
• Vinod Pamulapati | HPC Performance Engineer

Other contributors:

• Guy Bursell | Director Business Strategy
• Sachin Rastogi | Manager

To see non-public LinkedIn profiles, sign in to LinkedIn.

Next steps

• GPU-optimized VM sizes
• VMs on Azure
• Virtual networks and VMs on Azure
• Learning path: Run high-performance computing (HPC) applications on Azure

Related resources

• Run a Linux VM on Azure
• HPC system and big-compute solutions