Question about ETL (Azure Data Factory)

Sebastian Pacheco 181 Puntos de reputación
2024-06-24T12:56:57.72+00:00

Hello everyone... I have a question, I was reading about Azure Data Factory, Azure Databricks, Azure Synapse Analytics, Azure Data Lake and SQL Database and I am in doubt which one is best for me to use... My scheme is as follows:

I have Azure database for postgresql Flexible Server where I have my data, I need a tool that will search for certain information and then with python I can do calculations and then leave this new information in some storage where "Power Bi" can connect and create reports and graphs .

Based on this I was seeing that with ADF I could build a pipeline to rescue the data, pass it to Databricks so that the data analyst can program in python and do the calculations, then this is sent to synapse or data lake or sql database and connect " Power Bi" to this service to create reports and graphs.

My questions are:

1.- Would a scheme like this be good?

2.- ADF is strong for doing data transformation manually with python or is databricks better?

3.- Maybe at first we only need "storage" (and nothing else) to connect Power Bi... between synapse, sql databae or data lake, if this is the case with a SQL Databe would it be the most economical?

Thanks!

Azure
Azure
Plataforma e infraestructura de informática en la nube para crear, implementar y administrar aplicaciones y servicios a través de una red mundial de centros de datos administrados por Microsoft.
251 preguntas
0 comentarios No hay comentarios
{count} votos

1 respuesta

Ordenar por: Muy útil
  1. Sebastian Pacheco 181 Puntos de reputación
    2024-07-09T14:30:42.5033333+00:00

    Hello?? Has anyone worked with these tools? Can I just create a python script with ADF and do the data transformations/calculations?

    0 comentarios No hay comentarios