How to pass column list as argument from databricks spark for copy write semantic

Rishabh 11 Reputation points
2020-07-23T06:54:16.693+00:00

Is there a way to pass column list argument for column mapping between spark and synapse table from databricks spark for write semantics as copy as we pass it while running copy command from synapse?

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,369 questions
Azure Databricks
Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
1,920 questions

1 answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. HimanshuSinha-msft 19,376 Reputation points Microsoft Employee
    2020-07-27T23:33:22.18+00:00

    Hello Rishabh ,
    Thanks for the ask and also using the forum .

    As I understand the ask here is understand as to how column name ( with mapping) can be used in the copy command on Synapse .

    Below is the command and screenshot whcih should help you out with that . 

    COPY INTO NameofTheTable( NameOfTheSourceColumn IndexOfTheColumnOnTheSinkSide)

    13972-copy-command.png

    Please do let me know how it goes .

    Thanks & stay safe

    Himanshu

    Please do consider to click on "Accept Answer" and "Up-vote" on the post that helps you, as it can be beneficial to other community members

    0 comments