Azure Data Factory - MySQL ODBC Operation Failed (The Connection Has Been Disabled)

Goyal, Saurabh 11 Reputation points
2022-03-23T15:15:24.707+00:00

We keep continuing to get this error message when trying to copy a large set of data (around 45 million records) from MySQL Table to AzureSQL Database Table. Pipeline always starts fine and run for the first couple of minutes and then stops with the following error message:

Operation on target COPY-USAGE-DATA failed: Failure happened on 'Source' side. ErrorCode=UserErrorUnclassifiedError,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Odbc Operation Failed.,Source=Microsoft.DataTransfer.ClientLibrary.Odbc.OdbcConnector,''Type=System.InvalidOperationException,Message=The connection has been disabled.,Source=System.Data,''Type=System.Data.Odbc.OdbcException,Message=ERROR [08S01] [Microsoft][MySQL] (17) Error during communication with the server.,Source=MySQLODBC_sb64.dll,'

Here is the pipeline JSON for your reference:
{
"name": "Copy_WebService_Usage",
"properties": {
"description": "Copy Web Service API Log Tables",
"activities": [
{
"name": "COPY-USAGE-DATA",
"description": "Copy Usage Log Data",
"type": "Copy",
"dependsOn": [],
"policy": {
"timeout": "7.00:00:00",
"retry": 0,
"retryIntervalInSeconds": 30,
"secureOutput": false,
"secureInput": false
},
"userProperties": [],
"typeProperties": {
"source": {
"type": "MySqlSource"
},
"sink": {
"type": "AzureSqlSink",
"preCopyScript": "TRUNCATE TABLE USAGELOG_2019",
"writeBehavior": "insert",
"sqlWriterUseTableLock": false,
"tableOption": "autoCreate",
"disableMetricsCollection": false
},
"enableStaging": false,
"translator": {
"type": "TabularTranslator",
"typeConversion": true,
"typeConversionSettings": {
"allowDataTruncation": true,
"treatBooleanAsNumber": false
}
}
},
"inputs": [
{
"referenceName": "SourceMysqlDB",
"type": "DatasetReference"
}
],
"outputs": [
{
"referenceName": "TargetAzureLSqlDB",
"type": "DatasetReference"
}
]
}
],
"annotations": [],
"lastPublishTime": "2022-03-22T19:39:15Z"
},
"type": "Microsoft.DataFactory/factories/pipelines"
}

Any idea why we are getting this error, I have tested all the connection settings and MySQL (Source Database) and it's up and running fine.

Azure Database for MySQL
Azure Database for MySQL
An Azure managed MySQL database service for app development and deployment.
714 questions