Integration with ArcGis

Question

I am trying to integrate various shape files from ArcGis source database to Azure. While exploring I understand that cant connect these files using Azure Data factory and can use Python scripts (GeoPandas) can be able to read these files from ArcGis. My question was

1. How these files can be stored in Azure ? which format will be efficient for geospatial analysis
2. Is there any other way that can be used to integrate the ArcGis Source apart from Python scripts?

Answer 1

1. The efficient format for storing geospatial data in Azure depends on the specific use case and requirements. The architecture flow described in the provided context suggests that after data ingestion, it can be stored permanently in warm storage with Azure SQL, Azure SQL Managed Instance, Azure Database for PostgreSQL, or Azure Data Lake Storage. From there, the data can be transformed and processed in batch with Azure Batch or Synapse Spark Pool, both of which can be automated through the usage of an Azure Data Factory or Synapse pipeline. For real-time data, it can be further transformed or processed with Stream Analytics, Azure Maps, or brought into context with Azure Digital Twins. Once the data is transformed, it can then once again be served for additional uses in Azure SQL DB or Azure Database for PostgreSQL, Synapse SQL Pool (for abstracted non-geospatial data), Azure Cosmos DB, or Azure Data Explorer. The Azure Maps Data API would suffice for small datasets, otherwise, a non-native service can be introduced based on OSS or COTS, for accessing the data through web services or desktop applications. Finally, the Azure Maps Web SDK hosted in Azure App Service would allow for geovisualization. Another option is to use Azure Maps in Power BI.
2. Apart from Python scripts, there are various Spark libraries available for working with geospatial data on Azure Databricks and Synapse Spark Pools. Some of these libraries include Apache Sedona (GeoSpark), GeoPandas, GeoTrellis, PySAL, Rasterio, WhiteboxTools, Turf.js, Pointpats, Raster Vision, EarthPy, Planetary Computer, and PDAL. Additionally, vector tiles provide an efficient way to display GIS data on maps, and a solution could use PostGIS to dynamically query vector tiles.

---

References:

• End-to-end geospatial storage, analysis, and visualization - General geospatial architecture
• End-to-end geospatial storage, analysis, and visualization - Alternatives