An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
Need to have more clearance spark synapse copy write options
Hi All,
For my business requirements, my spark code in databricks were used to load a gigabytes of data into Azure synapse database.
The actual issue was by using COPY Into I can able to use option errorfile used to store error records.
But in spark I am unable to achieve it. Could any one help us to achieve this.
Spark code:
spark.conf.set("fs.azure.account.auth.type", "OAuth")
spark.conf.set("fs.azure.account.oauth.provider.type", "org.apache.hadoop.fs.azurebfs.oauth2.ClientCredsTokenProvider")
spark.conf.set("fs.azure.account.oauth2.client.id", "
Azure Synapse Analytics
Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
Azure Dedicated Host
An Azure service that provides a dedicated physical server to host Azure virtual machines for Windows and Linux.