Rearchitect the event-driven workflow (EDW) workload for Azure Kubernetes Service (AKS)
Now that you understand some key platform differences between AWS and Azure relevant to this workload, let's take a look at the workflow architecture and we can change it to work on AKS.
AWS workload architecture
The AWS workload is a basic example of the competing consumers design pattern. The AWS implementation is a reference architecture for managing scale and cost for event-driven workflows using Kubernetes, Kubernetes Event-driven Autoscaling (KEDA), and Karpenter.
A producer app generates load through sending messages to a queue, and a consumer app running in a Kubernetes pod processes the messages and writes the results to a database. KEDA manages pod autoscaling through a declarative binding to the producer queue, and Karpenter manages node autoscaling with just enough compute to optimize for cost. Authentication to the queue and the database uses OAuth-based service account token volume projection.
The workload consists of an AWS EKS cluster to orchestrate consumers reading messages from an Amazon Simple Queue Service (SQS) and saving processed messages to an Amazon DynamoDB table. A producer app generates messages and queues them in the Amazon SQS queue. KEDA and Karpenter dynamically scale the number of EKS nodes and pods used for the consumers.
The following diagram represents the architecture of the EDW workload in AWS:
Map AWS services to Azure services
To recreate the AWS workload in Azure with minimal changes, use an Azure equivalent for each AWS service and keep authentication methods similar to the original. This example doesn't require the advanced features of Azure Service Bus or Azure Event Hubs. Instead, you can use Azure Queue Storage to queue up work, and Azure Table storage to store results.
The following table summarizes the service mapping:
| Service mapping | AWS service | Azure service |
|---|---|---|
| Queuing | Simple Queue Service | Azure Queue Storage |
| Persistence | DynamoDB (No SQL) | Azure Table storage |
| Orchestration | Elastic Kubernetes Service (EKS) | Azure Kubernetes Service (AKS) |
| Identity | AWS IAM | Microsoft Entra |
Azure workload architecture
The following diagram represents the architecture of the Azure EDW workload using the AWS to Azure service mapping:
Compute options
Depending on cost considerations and resilience to possible node eviction, you can choose from different types of compute.
In AWS, you can choose between on-demand compute (more expensive but no eviction risk) or Spot instances (cheaper but with eviction risk). In AKS, you can choose an on-demand node pool or a Spot node pool depending on your workload's needs.
Next steps
Contributors
Microsoft maintains this article. The following contributors originally wrote it:
- Ken Kilty | Principal TPM
- Russell de Pina | Principal TPM
- Jenny Hayes | Senior Content Developer
- Carol Smith | Senior Content Developer
- Erin Schaffer | Content Developer 2
Azure Kubernetes Service