Why does importing data to my CosmosDB with Data Migration Tool on Mac fail?

ASCIImo 0 Reputation points
2024-02-02T16:07:57.8833333+00:00

I'm trying to import JSON data into my CosmosDB on Azure with Data Migration Tool on Mac. The import goes smoothly at first, displaying 500... 1000... 4500 processed records, and then suddenly fails with the output below. I can see 4500+ records in the database, but not all included within the JSON. The JSON file passes validation, so I don't expect errors there. Terminal output:

fail: Cosmos.DataTransfer.Core.RunCommand.CommandHandler[0]

      Data transfer failed

      System.AggregateException: One or more errors occurred. (Response status code does not indicate success: Conflict (409); Substatus: 0; ActivityId: 37a57a39-c43c-4d5e-b088-a3f21d428273; Reason: ();)

       ---> Microsoft.Azure.Cosmos.CosmosException : Response status code does not indicate success: Conflict (409); Substatus: 0; ActivityId: 37a57a39-c43c-4d5e-b088-a3f21d428273; Reason: ();

         at Microsoft.Azure.Cosmos.ResponseMessage.EnsureSuccessStatusCode()

         at Microsoft.Azure.Cosmos.CosmosResponseFactoryCore.ProcessMessage[T](ResponseMessage responseMessage, Func`2 createResponse)

         at Microsoft.Azure.Cosmos.ContainerCore.CreateItemAsync[T](T item, ITrace trace, Nullable`1 partitionKey, ItemRequestOptions requestOptions, CancellationToken cancellationToken)

         at Microsoft.Azure.Cosmos.ClientContextCore.RunWithDiagnosticsHelperAsync[TResult](String containerName, String databaseName, OperationType operationType, ITrace trace, Func`2 task, Func`2 openTelemetry, String operationName, RequestOptions requestOptions)

         at Microsoft.Azure.Cosmos.ClientContextCore.OperationHelperWithRootTraceAsync[TResult](String operationName, String containerName, String databaseName, OperationType operationType, RequestOptions requestOptions, Func`2 task, Func`2 openTelemetry, TraceComponent traceComponent, TraceLevel traceLevel)

         at Cosmos.DataTransfer.CosmosExtension.CosmosDataSinkExtension.PopulateItem(Container container, ExpandoObject item, String partitionKeyPath, DataWriteMode mode, String itemId, CancellationToken cancellationToken) in /__w/data-migration-desktop-tool/data-migration-desktop-tool/Extensions/Cosmos/Cosmos.DataTransfer.CosmosExtension/CosmosDataSinkExtension.cs:line 190

         at Polly.Retry.AsyncRetryEngine.ImplementationAsync[TResult](Func`3 action, Context context, CancellationToken cancellationToken, ExceptionPredicates shouldRetryExceptionPredicates, ResultPredicates`1 shouldRetryResultPredicates, Func`5 onRetryAsync, Int32 permittedRetryCount, IEnumerable`1 sleepDurationsEnumerable, Func`4 sleepDurationProvider, Boolean continueOnCapturedContext)

         at Polly.AsyncPolicy.ExecuteAsync[TResult](Func`3 action, Context context, CancellationToken cancellationToken, Boolean continueOnCapturedContext) 
      --- Cosmos Diagnostics ---

Then JSON code as in the attached file (too long to insert in code block and maintain readability): dmt_output.txt. And finally:

         --- End of inner exception stack trace ---

         at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)

         at System.Threading.Tasks.Task`1.GetResultCore(Boolean waitCompletionNotification)

         at Cosmos.DataTransfer.CosmosExtension.CosmosDataSinkExtension.<>c__DisplayClass4_0.<AddItemAsync>b__1(Task`1 t) in /__w/data-migration-desktop-tool/data-migration-desktop-tool/Extensions/Cosmos/Cosmos.DataTransfer.CosmosExtension/CosmosDataSinkExtension.cs:line 154

         at System.Threading.Tasks.ContinuationResultTaskFromResultTask`2.InnerInvoke()

         at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)

      --- End of stack trace from previous location ---

         at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot, Thread threadPoolThread)

      --- End of stack trace from previous location ---

         at Cosmos.DataTransfer.CosmosExtension.CosmosDataSinkExtension.WriteAsync(IAsyncEnumerable`1 dataItems, IConfiguration config, IDataSourceExtension dataSource, ILogger logger, CancellationToken cancellationToken) in /__w/data-migration-desktop-tool/data-migration-desktop-tool/Extensions/Cosmos/Cosmos.DataTransfer.CosmosExtension/CosmosDataSinkExtension.cs:line 103

         at Cosmos.DataTransfer.CosmosExtension.CosmosDataSinkExtension.WriteAsync(IAsyncEnumerable`1 dataItems, IConfiguration config, IDataSourceExtension dataSource, ILogger logger, CancellationToken cancellationToken) in /__w/data-migration-desktop-tool/data-migration-desktop-tool/Extensions/Cosmos/Cosmos.DataTransfer.CosmosExtension/CosmosDataSinkExtension.cs:line 99

         at Cosmos.DataTransfer.Core.RunCommand.CommandHandler.ExecuteDataTransferOperation(IDataSourceExtension source, IConfiguration sourceConfig, IDataSinkExtension sink, IConfiguration sinkConfig, CancellationToken cancellationToken) in /__w/data-migration-desktop-tool/data-migration-desktop-tool/Core/Cosmos.DataTransfer.Core/RunCommand.cs:line 179 

What might be wrong?

Azure Cosmos DB
Azure Cosmos DB
An Azure NoSQL database service for app development.
1,901 questions
{count} votes

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.