Database experimentation Assistant (DEA)- Replay trace facing issue.

Nikunj Patel 186 Reputation points
2020-06-26T12:04:39.557+00:00

We are measuring SQL server performance between Azure MI SQL server 2012 and 2019 using DEA. In DEA we have successfully save XEvent on blob storage. Now when we are trying to replay trace on Target A and when replaytrace is running we are getting replay status as error and it stop to run the trace on target.

We used following configuration in replay trace details.

10812-replay.png

After that when we have check log from %temp%/DEA then we are get following log data. But we cant find any solution from this log. Can anyone have any idea how we can run replay trace using DEA on Azure MI.

DEA Information: 0 : DiagnosticEvent, 06/26/2020 11:34:54, Data: {"Message":"Entered GetSqlServerConnectionMetaData","Timestamp":"2020-06-26T11:34:54.1045831Z"}
DEA Information: 0 : DiagnosticEvent, 06/26/2020 11:34:54, Data: {"Message":"Attempting to get the SQL server metadata.","Operation":"ReplayModel","Properties":{"ServerRoleName":null,"ProductVersion":null,"TraceId":"0","SelectedDuration":"0","TraceType":"Extensive","ControllerStatus":"INITIALIZED","CaptureStatus":"NotStarted","PreprocessTraceStatus":"NotStarted","ReplayStatus":"NotStarted","CaptureStartTime":"1/1/0001 12:00:00 AM","CaptureStopTime":"1/1/0001 12:00:00 AM","CaptureDuration":"00:00:00","ErrorCode":"0","Id":"1d411298-e5cb-4850-ae9b-2445c042ab9e"},"Timestamp":"2020-06-26T11:34:54.1055805Z"}
DEA Information: 0 : TraceEvent, 06/26/2020 11:34:59, Data: {"Message":"Successfully retrieved SQL server metadata.","Operation":"ReplayModel","Properties":{"ServerRoleName":"sysadmin","ProductVersion":"12.0.2000.8","TraceId":"0","SelectedDuration":"0","TraceType":"Extensive","ControllerStatus":"INITIALIZED","CaptureStatus":"NotStarted","PreprocessTraceStatus":"NotStarted","ReplayStatus":"NotStarted","CaptureStartTime":"1/1/0001 12:00:00 AM","CaptureStopTime":"1/1/0001 12:00:00 AM","CaptureDuration":"00:00:00","ErrorCode":"0","Id":"1d411298-e5cb-4850-ae9b-2445c042ab9e"},"Timestamp":"2020-06-26T11:34:59.360549Z"}
DEA Information: 0 : MetricEvent, 06/26/2020 11:34:59, Data: {"IsSuccess":false,"Name":"Duration","StartTime":"2020-06-26T11:34:54.1045831Z","Value":5258.6812,"Operation":"GetSqlServerConnectionMetaData","Timestamp":"2020-06-26T11:34:59.3635393Z"}
DEA Information: 0 : DiagnosticEvent, 06/26/2020 11:34:59, Data: {"Message":"Exited GetSqlServerConnectionMetaData","Timestamp":"2020-06-26T11:34:59.3644834Z"}
DEA Information: 0 : DiagnosticEvent, 06/26/2020 11:34:59, Data: {"Message":"Entered VerifyUserServerRoleName","Timestamp":"2020-06-26T11:34:59.3664745Z"}
DEA Information: 0 : MetricEvent, 06/26/2020 11:34:59, Data: {"IsSuccess":false,"Name":"Duration","StartTime":"2020-06-26T11:34:59.3674734Z","Value":2.5428,"Operation":"VerifyUserServerRoleName","Timestamp":"2020-06-26T11:34:59.36947Z"}
DEA Information: 0 : DiagnosticEvent, 06/26/2020 11:34:59, Data: {"Message":"Exited VerifyUserServerRoleName","Timestamp":"2020-06-26T11:34:59.3714593Z"}
DEA Information: 0 : DiagnosticEvent, 06/26/2020 11:34:59, Data: {"Message":"Entered StartXEventsCapture","Timestamp":"2020-06-26T11:34:59.3726371Z"}
DEA Information: 0 : DiagnosticEvent, 06/26/2020 11:35:01, Data: {"Message":"Executing query for starting SQL XEvents on the database server.","Timestamp":"2020-06-26T11:35:01.6111895Z"}
DEA Information: 0 : MetricEvent, 06/26/2020 11:35:03, Data: {"IsSuccess":false,"Name":"Duration","StartTime":"2020-06-26T11:34:59.3746986Z","Value":3949.6299000000004,"Operation":"StartXEventsCapture","Timestamp":"2020-06-26T11:35:03.3239235Z"}
DEA Information: 0 : DiagnosticEvent, 06/26/2020 11:35:03, Data: {"Message":"Exited StartXEventsCapture","Timestamp":"2020-06-26T11:35:03.3249172Z"}
DEA Information: 0 : DiagnosticEvent, 06/26/2020 11:35:03, Data: {"Message":"Entered StopXEventsCapture","Timestamp":"2020-06-26T11:35:03.3259146Z"}
DEA Information: 0 : DiagnosticEvent, 06/26/2020 11:35:03, Data: {"Message":"Executing Stop Capture Trace Query on the Experiment database server 'dev-sqlmi-sun-eastus.e796c9768f7f.database.windows.net'","Timestamp":"2020-06-26T11:35:03.3289129Z"}
DEA Information: 0 : DiagnosticEvent, 06/26/2020 11:35:03, Data: {"Message":"Executing query for stopping SQL XEvents on the database server.","Timestamp":"2020-06-26T11:35:03.3299034Z"}
DEA Information: 0 : TraceEvent, 06/26/2020 11:35:03, Data: {"Message":"Triggered stop capture.","Operation":"ReplayModel","Properties":{"ServerRoleName":"sysadmin","ProductVersion":"12.0.2000.8","TraceId":"65544","SelectedDuration":"0","TraceType":"Extensive","ControllerStatus":"INITIALIZED","CaptureStatus":"Stopped","PreprocessTraceStatus":"NotStarted","ReplayStatus":"NotStarted","CaptureStartTime":"1/1/0001 12:00:00 AM","CaptureStopTime":"1/1/0001 12:00:00 AM","CaptureDuration":"00:00:00","ErrorCode":"0","Id":"1d411298-e5cb-4850-ae9b-2445c042ab9e"},"Timestamp":"2020-06-26T11:35:03.7409346Z"}
DEA Information: 0 : MetricEvent, 06/26/2020 11:35:03, Data: {"IsSuccess":false,"Name":"Duration","StartTime":"2020-06-26T11:35:03.3269109Z","Value":415.5421,"Operation":"StopXEventsCapture","Timestamp":"2020-06-26T11:35:03.7418721Z"}
DEA Information: 0 : DiagnosticEvent, 06/26/2020 11:35:03, Data: {"Message":"Exited StopXEventsCapture","Timestamp":"2020-06-26T11:35:03.7429592Z"}
DEA Information: 0 : DiagnosticEvent, 06/26/2020 11:35:03, Data: {"Message":"Entered PerformFileOperation","Timestamp":"2020-06-26T11:35:03.7438921Z"}
DEA Error: -2146233088 : ExceptionEvent, 06/26/2020 11:35:04, Data: {"Code":-2146233088,"Message":"Unable to empty blob container replayxevent","Timestamp":"2020-06-26T11:35:04.9314173Z"}, Exception:
[Exception Info]:Microsoft.WindowsAzure.Storage.StorageException, Message:The remote server returned an error: (403) Forbidden.
StackTrace: at Microsoft.WindowsAzure.Storage.Core.Executor.Executor.ExecuteSyncT
at Microsoft.WindowsAzure.Storage.Blob.CloudBlob.Delete(DeleteSnapshotsOption deleteSnapshotsOption, AccessCondition accessCondition, BlobRequestOptions options, OperationContext operationContext)
at Microsoft.WindowsAzure.Storage.Blob.CloudBlob.DeleteIfExists(DeleteSnapshotsOption deleteSnapshotsOption, AccessCondition accessCondition, BlobRequestOptions options, OperationContext operationContext)
at Microsoft.DEA.BusinessLogic.Actions.PerformFileOperation.TryEmptyBlobContainer()System.Net.WebException[InnerException]:
Message:The remote server returned an error: (403) Forbidden.
StackTrace: at System.Net.HttpWebRequest.GetResponse()
at Microsoft.WindowsAzure.Storage.Core.Executor.Executor.ExecuteSyncT
DEA Information: 0 : MetricEvent, 06/26/2020 11:35:04, Data: {"IsSuccess":false,"Name":"Duration","StartTime":"2020-06-26T11:35:03.7448902Z","Value":1206.9254,"Operation":"PerformFileOperation","Timestamp":"2020-06-26T11:35:04.9513269Z"}
DEA Information: 0 : DiagnosticEvent, 06/26/2020 11:35:04, Data: {"Message":"Exited PerformFileOperation","Timestamp":"2020-06-26T11:35:04.9523276Z"}
DEA Information: 0 : DiagnosticEvent, 06/26/2020 11:35:04, Data: {"Message":"Entered StartXEventsPreprocessing","Timestamp":"2020-06-26T11:35:04.953418Z"}
DEA Information: 0 : MetricEvent, 06/26/2020 11:35:04, Data: {"IsSuccess":false,"Name":"Duration","StartTime":"2020-06-26T11:35:04.9543227Z","Value":6.0375000000000005,"Operation":"StartXEventsPreprocessing","Timestamp":"2020-06-26T11:35:04.9603077Z"}
DEA Information: 0 : DiagnosticEvent, 06/26/2020 11:35:04, Data: {"Message":"Exited StartXEventsPreprocessing","Timestamp":"2020-06-26T11:35:04.9615082Z"}
DEA Information: 0 : TraceEvent, 06/26/2020 11:35:04, Data: {"Message":"Completed trigger of replay on the server.","Operation":"ReplayModel","Properties":{"ServerRoleName":"sysadmin","ProductVersion":"12.0.2000.8","TraceId":"65544","SelectedDuration":"0","TraceType":"Extensive","ControllerStatus":"INITIALIZED","CaptureStatus":"Stopped","PreprocessTraceStatus":"InProgress","ReplayStatus":"NotStarted","CaptureStartTime":"1/1/0001 12:00:00 AM","CaptureStopTime":"1/1/0001 12:00:00 AM","CaptureDuration":"00:00:00","ErrorCode":"0","Id":"1d411298-e5cb-4850-ae9b-2445c042ab9e"},"Timestamp":"2020-06-26T11:35:04.9625486Z"}
DEA Information: 0 : DiagnosticEvent, 06/26/2020 11:35:07, Data: {"Message":"Entered GetOStressProgress","Timestamp":"2020-06-26T11:35:07.0301633Z"}
DEA Information: 0 : MetricEvent, 06/26/2020 11:35:07, Data: {"IsSuccess":true,"Name":"Duration","StartTime":"2020-06-26T11:35:07.0310972Z","Value":16.4361,"Operation":"GetOStressProgress","Timestamp":"2020-06-26T11:35:07.0470221Z"}
DEA Error: -2146233088 : ExceptionEvent, 06/26/2020 11:35:07, Data: {"Code":-2146233088,"Message":"Exception thrown in Replay monitor for replay context.","Operation":"ReplayModel","Properties":{"ServerRoleName":"sysadmin","ProductVersion":"12.0.2000.8","TraceId":"65544","SelectedDuration":"0","TraceType":"Extensive","ControllerStatus":"PREPROCESSING","CaptureStatus":"Stopped","PreprocessTraceStatus":"Error","ReplayStatus":"Error","CaptureStartTime":"1/1/0001 12:00:00 AM","CaptureStopTime":"1/1/0001 12:00:00 AM","CaptureDuration":"00:00:00","ErrorCode":"1803","Id":"1d411298-e5cb-4850-ae9b-2445c042ab9e"},"Timestamp":"2020-06-26T11:35:07.052001Z"}, Exception:
[Exception Info]:Microsoft.DEA.Common.Exceptions.DeaException, Message:Exception of type 'Microsoft.DEA.Common.Exceptions.DeaException' was thrown.
StackTrace: at Microsoft.DEA.BusinessLogic.Actions.GetOStressProgress.Run()
at Microsoft.DEA.BusinessLogic.ActionDecorator.<>c__DisplayClass2_0.<Run>b__0()
at System.Threading.Tasks.Task.InnerInvoke()
at System.Threading.Tasks.Task.Execute()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.DEA.BusinessLogic.ActionDecorator.<Run>d__2.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at Microsoft.DEA.BusinessLogic.ActionDecorator.<Run>d__2.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.DEA.BusinessLogic.Bootstrappers.OStressReplayProgress.<RunAsync>d__6.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at Microsoft.DEA.BusinessLogic.Bootstrappers.OStressReplayProgress.<RunAsync>d__6.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.DEA.BusinessLogic.BootstrapperFactory.<StartWorkflowAsync>d__37.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.DEA.BusinessLogic.BootstrapperFactory.<CreateReplayProgressBootstrapperAsync>d__29.MoveNext()
DEA Information: 0 : DiagnosticEvent, 06/26/2020 11:36:09, Data: {"Message":"Entered CancelOStressOngoingTask","Timestamp":"2020-06-26T11:36:09.7667023Z"}
DEA Information: 0 : MetricEvent, 06/26/2020 11:36:09, Data: {"IsSuccess":false,"Name":"Duration","StartTime":"2020-06-26T11:36:09.7667023Z","Value":7.4542,"Operation":"CancelOStressOngoingTask","Timestamp":"2020-06-26T11:36:09.774681Z"}
DEA Information: 0 : DiagnosticEvent, 06/26/2020 11:36:09, Data: {"Message":"Exited CancelOStressOngoingTask","Timestamp":"2020-06-26T11:36:09.774681Z"}
DEA Information: 0 : DiagnosticEvent, 06/26/2020 11:36:09, Data: {"Message":"Entered StopXEventsCapture","Timestamp":"2020-06-26T11:36:09.774681Z"}
DEA Information: 0 : DiagnosticEvent, 06/26/2020 11:36:09, Data: {"Message":"Executing Stop Capture Trace Query on the Experiment database server 'dev-sqlmi-sun-eastus.e796c9768f7f.database.windows.net'","Timestamp":"2020-06-26T11:36:09.7756784Z"}
DEA Information: 0 : DiagnosticEvent, 06/26/2020 11:36:09, Data: {"Message":"Executing query for stopping SQL XEvents on the database server.","Timestamp":"2020-06-26T11:36:09.7756784Z"}
DEA Information: 0 : TraceEvent, 06/26/2020 11:36:10, Data: {"Message":"Triggered stop capture.","Operation":"ReplayModel","Properties":{"PreprocessTraceStatus":"Stopped","ReplayStatus":"Stopped","CaptureStartTime":"1/1/0001 12:00:00 AM","CaptureDuration":"00:00:00","ServerRoleName":"sysadmin","ProductVersion":"12.0.2000.8","TraceId":"65544","SelectedDuration":"0","TraceType":"Extensive","ControllerStatus":"PREPROCESSING","CaptureStatus":"Stopped","CaptureStopTime":"1/1/0001 12:00:00 AM","ErrorCode":"1803","Id":"1d411298-e5cb-4850-ae9b-2445c042ab9e"},"Timestamp":"2020-06-26T11:36:10.2158316Z"}
DEA Information: 0 : MetricEvent, 06/26/2020 11:36:10, Data: {"IsSuccess":false,"Name":"Duration","StartTime":"2020-06-26T11:36:09.7756784Z","Value":442.79810000000003,"Operation":"StopXEventsCapture","Timestamp":"2020-06-26T11:36:10.217718Z"}
DEA Information: 0 : DiagnosticEvent, 06/26/2020 11:36:10, Data: {"Message":"Exited StopXEventsCapture","Timestamp":"2020-06-26T11:36:10.2188015Z"}

Azure Database Migration service
{count} votes

Accepted answer
  1. Mike Ubezzi 2,776 Reputation points
    2020-06-29T18:16:12.473+00:00

    @NikunjPatel-2604 - Based upon the following single error from what you provided, there is an issue with the Azure Blob storage account access or permissions.

    DEA Error: -2146233088 : ExceptionEvent, 06/26/2020 11:35:04, Data: {"Code":-2146233088,"Message":"Unable to empty blob container replayxevent","Timestamp":"2020-06-26T11:35:04.9314173Z"}, Exception:

    [Exception Info]:Microsoft.WindowsAzure.Storage.StorageException, Message:The remote server returned an error: (403) Forbidden.

    There is a container in your Blob Storage account replayxevent that I think needs execute permissions enabled to the DEA process that is reading events from this location or the URI used for the Full path to source file and Replay Trace Location does not include the correct SAS URI. Please take a look at the message below the Replay Trace Location indicating that:

    Path to store the traces / xevents. For Azure SQL DB or Azure SQL Managed Instance account, you need to provide the Azure Blob storage account's SAS URI.

    So, you either don't have the correct SAS URL/URI or your SAS token does not include delete permissions.

    10899-screenshot-85.png

    10900-screenshot-86.png


0 additional answers

Sort by: Most helpful