Can you help me in bringing high transactional data from several On prem Sql servers to Fabric Onelake in real time without any intermediate storage

Question

I am having several SQL servers of 2019 version as source where the rate of transaction is high. From the servers i need to bring data to Fabric One-lake in real-time without using any scheduled pipelines. Data needs to be available in real time, there should be no intermediate storage, and there should be no impact on the source servers. Is is possible? Kindly help me.

Answer 1

Hi JLPM,

Thank you for reaching out to us on the Microsoft Q&A forum.

MS Fabric is currently not supported in the Q&A forums, the supported products are listed over here Supported products on Microsoft Q&A.

You can ask the experts in the dedicated MS Fabric community forum.

Can you please open as a new thread in MS Fabric community: General Discussion- Microsoft Fabric Community. Moderators are readily available there to assist you and provide guidance.

I hope this information is helpful, please Accept Answer & Upvote so that it would be helpful to community members.

Answer 2

Hello @JLPM

One possible solution to achieve this is by using Azure Stream Analytics. Azure Stream Analytics is a fully managed real-time data analysis service that allows you to process and analyze streaming data from various sources, including on-premises SQL servers, and output the results to various destinations, including Fabric One-lake.

To set up Azure Stream Analytics for your scenario, you can follow these 5 high-level steps:

1. Create an Azure Stream Analytics job in your Azure subscription.
2. Configure the input for the job to be the on-premises SQL servers that you want to replicate data from.
3. Configure the output for the job to be Fabric One-lake.
4. Write a Stream Analytics query that selects the data from the input and outputs it to the output in real-time.
5. Start the Stream Analytics job. By following these steps, you can replicate high transactional data from several on-premises SQL servers to Fabric One-lake in real-time without using any scheduled pipelines.

The data will be available in real-time, with no intermediate storage, and no impact on the source servers. Please note that you will need to ensure that your on-premises SQL servers are configured to allow outbound connections to Azure Stream Analytics.

You will also need to ensure that you have the necessary permissions to create and configure Azure Stream Analytics jobs and to access Fabric One-lake.

---

I hope that this response has addressed your query and helped you overcome your challenges. If so, please mark this response as Answered. This will not only acknowledge our efforts, but also assist other community members who may be looking for similar solutions.