This example architecture is built on the Basic enterprise integration architecture. It extends that architecture to show how to integrate enterprise backend systems, by using message brokers and events to decouple services for greater scalability and reliability. Ensure that you're familiar with that design and the components used in the basic integration architecture. It provides foundational information on the core components of this architecture, which won't be reproduced here.
Architecture
The backend systems referenced in this design may include software as a service (SaaS) systems, Azure services, and existing web services in your enterprise.
Download a Visio file of this architecture.
Workflow
The architecture shown here builds on a simpler architecture that is shown in Basic enterprise integration. That architecture uses Logic Apps to orchestrate workflows directly with backend systems and API Management to create catalogs of APIs.
This version of the architecture adds two components that help make the system more reliable and scalable:
Azure Service Bus. Service Bus is a secure, reliable message broker.
Azure Event Grid. Event Grid is an event routing service. It uses a publish/subscribe (pub/sub) eventing model.
Asynchronous communication using a message broker provides the following advantages over making direct, synchronous calls to backend services:
- Provides load-leveling to handle bursts in workloads, using the Queue-Based Load Leveling pattern.
- Provides for broadcasting of messages to multiple consumers using Publisher-Subscriber pattern.
- Reliably tracks the progress of long-running workflows that involve multiple steps or multiple applications.
- Helps to decouple applications.
- Integrates with existing message-based systems.
- Allows work to be queued when a backend system isn't available.
Event Grid enables the various components in the system to react to events as they happen, rather than relying on polling or scheduled tasks. As with a message queue and topics, it helps decouple applications and services. An application or service can publish events, and any interested subscribers will be notified. New subscribers can be added without updating the sender.
Many Azure services support sending events to Event Grid. For example, a logic app can listen for an event when new files are added to a blob store. This pattern enables reactive workflows, where uploading a file or putting a message on a queue kicks off a series of processes. The processes might be executed in parallel or in a specific sequence.
Recommendations
The recommendations described in Basic enterprise integration apply to this architecture.
Service Bus
Service Bus has two delivery modes, pull or proxied push. In the pull model, the receiver continuously polls for new messages. Polling can be inefficient, especially if you have many queues that each receive a few messages, or if there's much time between messages. In the proxied push model, Service Bus sends an event through Event Grid when there are new messages. The receiver subscribes to the event. When the event is triggered, the receiver pulls the next batch of messages from Service Bus.
When you create a logic app to consume Service Bus messages, we recommend using the proxied push model with Event Grid integration. It's often more cost efficient, because the logic app doesn't need to poll Service Bus. For more information, see Azure Service Bus to Event Grid integration overview. Currently, Service Bus Premium tier is required for Event Grid notifications.
Use PeekLock for accessing a group of messages. When you use PeekLock, the logic app can perform steps to validate each message before completing or abandoning the message. This approach protects against accidental message loss.
Event Grid
When an Event Grid trigger fires, it means at least one event happened. For example, when a logic app gets an Event Grid triggers for a Service Bus message, it should assume that several messages might be available to process.
Considerations
These considerations implement the pillars of the Azure Well-Architected Framework, which is a set of guiding tenets that can be used to improve the quality of a workload. For more information, see Microsoft Azure Well-Architected Framework.
Reliability
Reliability ensures your application can meet the commitments you make to your customers. For more information, see Overview of the reliability pillar.
- Microsoft Entra ID: Microsoft Entra ID is a globally distributed, highly available SaaS platform. Refer to the SLA for guaranteed availability details.
- API Management: API Management can be deployed in several highly available configurations, according to business requirements and cost tolerance. Refer to the Ensure API Management availability and reliability for a full review of options. Also refer to the SLA for guaranteed availability details.
- Logic Apps: Geo-redundant storage is available for Logic Apps on the Consumption plan tier. For information on designing a business continuity and disaster recovery solution, refer to the guidance. Also refer to the SLA for guaranteed availability details.
- Event Grid: Event Grid resource definitions for topics, system topics, domains, and event subscriptions and event data are automatically replicated across three availability zones (when available) in the region. When there's a failure in one of the availability zones, Event Grid resources automatically fail over to another availability zone without any human intervention. Refer to the Geo-disaster recovery across regions for guidance on designing a disaster recovery solution for failing over to another region. Also refer to the SLA for guaranteed availability details.
- Service Bus: Service Bus Premium supports Geo-disaster recovery and Availability Zones. Replication is available for Service Bus Standard. Also refer to the SLA for guaranteed availability details.
Security
Security provides assurances against deliberate attacks and the abuse of your valuable data and systems. For more information, see Overview of the security pillar.
To secure Service Bus, use Microsoft Entra authentication paired with managed identities. Microsoft Entra integration for Service Bus resources provides Azure role-based access control (RBAC) for fine-grained control over a client’s access to resources. You can use Azure RBAC to grant permissions to a security principal, which may be a user, a group, or an application service principal (a managed identity in this case).
Where Microsoft Entra ID isn't available, you can use shared access signature (SAS). You can grant a user access to Service Bus resources with specific rights by using SAS authentication.
If you need to expose a Service Bus queue or topic as an HTTP endpoint, for example, to post new messages, use API Management to secure the queue by fronting the endpoint. You can then secure the endpoint with certificates or OAuth authentication as appropriate. The easiest way to secure an endpoint is using a logic app with an HTTP request/response trigger as an intermediary.
The Event Grid service secures event delivery through a validation code. If you use Logic Apps to consume the event, validation is automatically performed. For more information, see Event Grid security and authentication.
Network security
Network security should be considered throughout the design.
- Service Bus Premium can be bound to a virtual network (VNet) subnet service endpoint, securing the namespace to only accept traffic from authorized virtual networks. Additionally, use private endpoints to lock down your VNet traffic to private traffic over Private Link.
- Logic Apps Standard and Premium Logic Apps can be configured to accept inbound traffic through private endpoints and to send outbound traffic through VNet integration.
- API Management provides several options to secure access to your API Management instance and APIs using an Azure virtual network. Refer to the Use a virtual network with Azure API Management documentation for a thorough review of the options. Private endpoints are also supported.
Cost Optimization
Cost optimization is about looking at ways to reduce unnecessary expenses and improve operational efficiencies. For more information, see Overview of the cost optimization pillar.
In general, use the Azure pricing calculator to estimate costs. Here are some other considerations.
API Management
You're charged for all API Management instances when they're running. If you've scaled up and don't need that level of performance all the time, manually scale down or configure autoscaling.
For light usage workloads, consider the consumption tier, which is a low-cost, serverless option. The consumption tier is billed per API call, whereas the other tiers are billed per hour.
Logic Apps
Logic Apps uses a serverless model. Billing is calculated based on action and connector execution. For more information, see Logic Apps pricing.
Service Bus queues, topics and subscriptions
Service Bus queues and subscriptions support both proxied push and pull models for delivering messages. In the pull model, every polling request is metered as an action. Even with long polling at 30 secs (default), cost can be high. Unless you need real-time delivery of messages, consider using the proxied push model.
Service Bus queues are included in all tiers (Basic, standard, and premium tiers). While Service Bus topics and subscriptions are available in standard and premium tiers. For more information, see Azure Service Bus pricing.
Event Grid
Event Grid uses a serverless model. Billing is calculated based on the number of operations (event executions). Operations include ingress of events to Domains or Topics, advanced matches, delivery attempts, and management calls. Usage of up to 100,000 operations is free of charge.
For more information, see Event Grid pricing.
For more information, see the cost section in Microsoft Azure Well-Architected Framework.
Operational Excellence
The Basic Enterprise Integration reference architecture provides guidance on DevOps patterns, which align to the Well-Architected Framework's Operational Excellence pillar.
Automating recovery operations as much as possible is an integral component of Operational Excellence. With automation in mind, you can combine Azure Log Monitoring with Azure Automation to automate the failover of your Service Bus resources. Refer to the diagram in failover flow documentation for an example of automation logic to initiate a failover.
Performance efficiency
Performance efficiency is the ability of your workload to scale to meet the demands placed on it by users in an efficient manner. For more information, see Performance efficiency pillar overview.
To achieve higher scalability, the Service Bus Premium tier can scale out the number of messaging units. Refer to the Service Bus Premium and Standard messaging tiers documentation for a review of the Premium tier benefits. Also, refer to the autoscaling feature documentation to learn about configuring the autoscaling of messaging units.
More recommendations for Service Bus can be found at Best practices for performance improvements by using Service Bus Messaging.
Next steps
For more information, see the Service Bus documentation:
- Azure Service Bus to Event Grid integration overview
- Service Bus Premium and Standard messaging tiers
- Service Bus autoscaling feature
- Best practices for performance improvements by using Service Bus Messaging