how connect to a source system - SAP database?

Question

how connect to a source SAP database?This project involves the migration of the existing data warehouse environment from Qlik to Microsoft Azure. The initiative aims to modernize the analytics platform by leveraging Azure’s scalable storage, compute, and data integration services, ensuring improved performance, enhanced security, and long‑term operational efficiency.

Answer 1

I think the only stable path is adf→sap thru a self‑hosted ir w/ nco… if a tiny‑table pull won’t pass, reinstall nco + rebind the ir, that fixes it.

Answer 2

Hi Makama, ANTHEA (404),
It sounds like you’re looking to extract data from an SAP source system into Azure as part of your Qlik‑to‑Azure modernization. For this scenario, Microsoft recommends using Azure Data Factory (ADF) or Data Factory in Microsoft Fabric, which provide native and supported SAP connectors for both full and incremental data movement.

Below is a practical, Microsoft‑aligned approach you can follow.
Overview of the recommended approach

Azure Data Factory acts as the integration layer that securely connects to SAP and moves data into Azure analytics services such as Azure Data Lake Storage Gen2, Azure Synapse Analytics, or Microsoft Fabric.

The exact configuration depends on your SAP system type (ECC, S/4HANA, BW, HANA) and whether you need full extracts or incremental (delta) loads.

1. Choose the appropriate SAP connector

Azure Data Factory supports multiple SAP connectors, each designed for a specific scenario:

• SAP Table connector – best suited for one‑time or periodic full extracts from SAP tables, views, or CDS views.
• SAP CDC connector – recommended for large datasets and ongoing replication, as it uses SAP’s Operational Data Provisioning (ODP) framework to capture deltas efficiently.
• SAP integration through Azure Logic Apps – suitable if the requirement involves RFC/BAPI calls or IDoc‑based integration rather than analytical data extraction.

These connectors are fully supported for SAP ECC, SAP S/4HANA, SAP BW, and SAP HANA environments.
2. Deploy a Self‑Hosted Integration Runtime (SHIR)

To connect securely to SAP, a Self‑Hosted Integration Runtime (SHIR) is required:

• Install the SHIR on a server (on‑premises or Azure VM) that has network access to the SAP system.
• If SAP is on‑premises, ensure VPN or ExpressRoute connectivity to Azure.
• Confirm that required SAP RFC ports (for example, 32xx or 33xx) are allowed through the firewall.

This runtime provides the secure bridge between Azure Data Factory and your SAP landscape.
3. Install the SAP .NET Connector on the SHIR host

For SAP Table and SAP CDC connectors to work, Microsoft requires the 64‑bit SAP Connector for Microsoft .NET (NCo 3.0):

• Download it from SAP Support Portal.
• Install it on the same machine where SHIR is running.
• Ensure the assemblies are registered in the Global Assembly Cache (GAC).

This step is mandatory and often the root cause of connectivity issues when skipped.
4. Create the SAP linked service in Azure Data Factory

In Azure Data Factory (or Fabric Data Factory):

• Create a new linked service using either SAP Table or SAP CDC connector.
• Provide SAP host or message server details, system number, client ID, credentials, or SNC configuration.
• Select the Self‑Hosted Integration Runtime and validate the connection.

Once the linked service is successfully tested, ADF can communicate with the SAP system.
5. Build pipelines for data extraction

Depending on your requirement:

• Use Copy Activity for full or batch table copies into Azure Data Lake Storage Gen2.
• Use SAP CDC with Mapping Data Flows for incremental replication, staging raw data first and then loading curated data into Synapse or Fabric.
• This approach minimizes load on SAP and supports scalable analytics.

6. Monitor and troubleshoot

After pipelines are running:

• Monitor pipeline executions and SHIR logs in Azure Data Factory.
• Verify SAP authorizations for RFC execution and ODP usage.
• For CDC scenarios, monitor delta queues in SAP using ODQMON to ensure changes are being consumed correctly.

By using Azure Data Factory with the appropriate SAP connector, along with a properly configured Self‑Hosted Integration Runtime and SAP prerequisites, you can reliably connect to your SAP source system and migrate data from Qlik to Azure. This approach follows Microsoft best practices, scales well for enterprise workloads, and supports both full and incremental data movement for modern analytics.

Microsoft Reference Documentation

• SAP connectors in Azure Data Factory https://learn.microsoft.com/azure/data-factory/industry-sap-connectors
• SAP Table connector prerequisites and setup https://learn.microsoft.com/azure/data-factory/connector-sap-table
• Prerequisites and configuration for SAP CDC connector https://learn.microsoft.com/azure/data-factory/sap-change-data-capture-prerequisites-configuration
• End‑to‑end SAP CDC into Azure / OneLake https://learn.microsoft.com/azure/data-factory/change-data-capture-from-sap-to-onelake-with-azure-data-factory
• Self‑Hosted Integration Runtime troubleshooting https://learn.microsoft.com/azure/data-factory/self-hosted-integration-runtime-troubleshoot-guide
• Azure Logic Apps SAP connector overview https://learn.microsoft.com/azure/logic-apps/logic-apps-using-sap-connector

Hope this helps. If you have any follow-up questions, please let me know. I would be happy to help.

Please do not forget to "Accept Answer" and "up-vote" wherever the information provided helps you, as this can be beneficial to other community members.

Answer 3

To connect Azure Data Factory (ADF) to a source SAP system as part of a migration to Azure, follow these high‑level steps from the provided guidance.

1. Prepare Azure and SAP prerequisites

• Collect SAP system details:
  • Hostname or IP address
  • System number
  • System ID (SID)
  • SAP user credentials
• In the Azure subscription, ensure the following resource providers are registered:
  • Microsoft.Storage
  • Microsoft.KeyVault
  • Microsoft.DataFactory This is done in Azure portal → Subscription → Resource providers.

2. Set up self-hosted Integration Runtime (for SAP application systems)

• For SAP ERP/S/4HANA sources using the SAP CDC connector, install a self-hosted integration runtime (SHIR) on a machine that can reach the SAP system.
• On that SHIR machine, download and install the SAP .NET Connector so ADF can connect via SAP RFC.
• Detailed SHIR and SAP .NET Connector setup is described in the SAP CDC connector preparation guidance.

3. Create the SAP linked service in Azure Data Factory

• In ADF, create a new linked service for SAP:
  • For SAP CDC scenarios, use the SAP CDC linked service type.
  • Provide SAP connection parameters: application/message server, instance number (system number), client ID, and SAP user credentials.
• The SAP user must be configured in SAP according to the SAP CDC prerequisites (authorizations, RFC, etc.).
• Creating a separate SAP CDC dataset is optional if using mapping data flows with inline datasets.

4. Configure SAP as a source system in Business Process Solutions (optional but common for SAP analytics)

• In the Business Process Solution item (for SAP analytics on Azure):
  1. Select Configure source system → New source system.
  2. Choose Connection type: Azure Data Factory, then select subscription, location, and specify a resource group name.
  3. In System Connection, enter the SAP system connection details and the service principal secret created as part of prerequisites.
  4. Click Create and monitor deployment. After completion, the ADF and related resources are deployed and wired to the SAP source.

5. Configure staging and downstream storage

• For CDC or batch ingestion, configure an Azure Data Lake Storage Gen2 linked service in ADF.
• SAP data is typically staged in ADLS Gen2, then transformed (for example, via mapping data flows) and loaded into the target Azure data warehouse or lakehouse.

6. Use in pipelines and data flows

• In ADF pipelines, use copy activities or mapping data flows with the SAP linked service as the source and ADLS Gen2 or other Azure targets as sinks.
• For change data capture from SAP into Fabric OneLake or a modern Azure analytics platform, use the provided SAP CDC templates and mapping data flows to merge changes into target tables.

These steps provide an end-to-end path: SAP system → SHIR with SAP .NET Connector → SAP linked service in ADF → optional Business Process Solution source system configuration → ADLS Gen2 staging → Azure analytics targets.

---

References:

• Change data capture from SAP to Microsoft Fabric OneLake with Azure Data Factory
• Configure SAP source system with Azure Data Factory
• SAP data integration example architecture