Hi, I have a real time data ingestion requirement from on prem oracle db to adls. In oracle db, we have pipeline functions and views where most of the business logic resides and I want to use them to ingest data in adls

Question

Hi, I have a real time data ingestion requirement from on prem oracle db to adls. In oracle db, we have pipeline functions and views where most of the business logic resides and I want to use them to ingest data in adls.

Kindly suggest, Thanks in advance

Answer 1

@Kapoor,Saumya,CH-Vevey Thanks for using Microsoft Q&A forum and posting your query.

You can explore Azure Data Factory or Azure Synapse Analytics for data ingestion from your Oracle to ADLS Gen2 but they are not real time. These services are better suited for batch processing.

If your requirement is to do real time data ingestion from your Oracle DB to ADLS Gen2 then, you may have to consider other applications like OCI GoldenGate

Here is a blog with detailed explanation on how to use OCI GoldenGate for real time ingestion of data to ADLS Gen2 - OCI GoldenGate Real-Time Ingestion for Azure Data Lake Storage

Related similar forum conversations:

1. Near real-time ETL of Oracle data to Azure SQL
2. Real Time Data Copy from Oracle DB to ADLS via Azure Data Factory?

Hope this info helps. Let us know if you have any questions.

---

Please don’t forget to Accept Answer and Yes for "was this answer helpful" wherever the information provided helps you, this can be beneficial to other community members.

Answer 2

@Kapoor,Saumya,CH-Vevey Thank you for your response. Unfortunately, there is not direct service in Azure which can identify the Oracle Realtime changes (or CDC feature) and process the data real-time. But as per your description, you have already implemented a solution using the ADF product, which is a very close solution but not an actual real time processing.

As I called out in my previous response, it would better to look for other third-party solutions for this requirement.

I would also highly recommend you please a feature request in Azure feedback forum and please do share the link once it is posted so that other users with similar idea can up-vote and comment on your feature request idea, which would help increase the priority of the feature request.

Here is the IDEAS forum link: https://feedback.azure.com/d365community/

Hope this info helps.

Thank you