Error when transferring data from Azure Synapse Serverless pool to Azure SQL DB

Priya Jha 866 Reputation points
2022-08-26T14:00:09.483+00:00

I am trying to copy data from tables present in Azure Synapse Serverless pool to Azure SQL DB using copy activity of ADF and receiving this below error intermittently for any table:

Failure happened on 'Source' side. ErrorCode=SqlOperationFailed,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=A database operation failed with the following error: 'Cannot bulk load because the file "https://***stg.dfs.core.windows.net/*******/2015.csv" is incomplete or could not be read. Operating system error code 12(The access code is invalid.).

The pipeline works for some tables and then fails for those same tables with the above error.

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,328 questions
{count} votes

Accepted answer
  1. Nandan Hegde 29,886 Reputation points MVP
    2022-09-06T15:53:29.29+00:00

    @AnnuKumari-MSFT , I guess based on the error message and the statement that this are intermittent errors, it seems the issue is similar to the issue in this thread:
    https://learn.microsoft.com/en-us/answers/questions/812041/pipeline-fails-because-of-file-lock-out.html

    which seems a bug at MSFT end.

    Can we get a confirmation from MSFT product team whether this issue has been fixed.

    1 person found this answer helpful.

1 additional answer

Sort by: Most helpful
  1. AnnuKumari-MSFT 30,361 Reputation points Microsoft Employee
    2022-08-29T05:32:17.21+00:00

    Hi @Priya Jha ,

    Thankyou for using Microsoft Q&A platform and thanks for posting your question.

    I understand while copying the data from Azure Serverless SQL pool to Azure SQL db, you are facing error for few tables.

    Could you please confirm when you say Serverless SQL pool tables, are you talking about external tables as we can't create any physical table in serverless sql pool. Also, For clarification, could you please share the screenshot of the pipeline . Thankyou in advance for your effort in doing so.

    Also, kindly try the below suggested workaround:

    Kindly create a database scoped credential and refer it in the external data source . Reference document: Database-scoped credential

    -- Optional: Create MASTER KEY if not exists in database:
    -- CREATE MASTER KEY ENCRYPTION BY PASSWORD = '<Very Strong Password>
    CREATE DATABASE SCOPED CREDENTIAL SynapseIdentity
    WITH IDENTITY = 'Managed Identity';
    GO
    CREATE EXTERNAL DATA SOURCE mysample
    WITH ( LOCATION = 'https://<storage_account>.dfs.core.windows.net/<container>/<path>',
    CREDENTIAL = SynapseIdentity
    )

    Hope this will help. Please let us know if any further queries.

    ------------------------------

    • Please don't forget to click on 130616-image.png or upvote 130671-image.png button whenever the information provided helps you.
      Original posters help the community find answers faster by identifying the correct answer. Here is how
    • Want a reminder to come back and check responses? Here is how to subscribe to a notification
    • If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators