Replicate an AWS event-driven workflow (EDW) workload with KEDA and Karpenter in Azure Kubernetes Service (AKS)

In this article, you learn how to replicate an Amazon Web Services (AWS) Elastic Kubernetes Service (EKS) event-driven workflow (EDW) workload with KEDA and Karpenter in AKS.

This workload is an implementation of the competing consumers pattern using a producer/consumer app that facilitates efficient data processing by separating data production from data consumption. You use KEDA to scale pods running consumer processing and Karpenter to autoscale Kubernetes nodes.

For a more detailed understanding of the AWS workload, see Scalable and Cost-Effective Event-Driven Workloads with KEDA and Karpenter on Amazon EKS.

Deployment process

1. Understand the conceptual differences: Start by reviewing the differences between AWS and AKS in terms of services, architecture, and deployment.
2. Rearchitect the workload: Analyze the existing AWS workload architecture and identify the components or services that you need to redesign to fit AKS. You need to make changes to the workload infrastructure, application architecture, and deployment process.
3. Update the application code: Ensure your code is compatible with Azure APIs, services, and authentication models.
4. Prepare for deployment: Modify the AWS deployment process to use the Azure CLI.
5. Deploy the workload: Deploy the replicated workload in AKS and test the workload to ensure that it functions as expected.

Prerequisites

• An Azure account. If you don't have one, create a free account before you begin.
• The Owner Azure built-in role, or the User Access Administrator and Contributor built-in roles, on a subscription in your Azure account.
• Azure CLI version 2.56 or later.
• Azure Kubernetes Service (AKS) preview extension.
• jq version 1.5 or later.
• Python 3 or later.
• kubectl version 1.21.0 or later
• Helm version 3.0.0 or later
• Visual Studio Code or equivalent.

Download the Azure application code

The completed application code for this workflow is available in our GitHub repository. Clone the repository to a directory called aws-to-azure-edw-workshop on your local machine by running the following command:

git clone https://github.com/Azure-Samples/aks-event-driven-replicate-from-aws ./aws-to-azure-edw-workshop

After you clone the repository, navigate to the aws-to-azure-edw-workshop directory and start Visual Studio Code by running the following commands:

cd aws-to-azure-edw-workshop
code .

Next steps

Understand platform differences

Contributors

This article is maintained by Microsoft. It was originally written by the following contributors:

• Ken Kilty | Principal TPM
• Russell de Pina | Principal TPM
• Jenny Hayes | Senior Content Developer
• Carol Smith | Senior Content Developer
• Erin Schaffer | Content Developer 2