Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
11,624 questions
Can not turn on Dataflow Debug in Azure Data Factory
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(99.2, 13%, 19%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKW%3C/text%3E%3C/svg%3E)
Kyle West
0
Reputation points
I am unable to toggle on the dataflow debug button. Here is the full error. Support is appreciated.
Creating debug session failed: Failed to setup debug session while checking out from debug pool. Session Id: eae39d9c-41b9-4724-ae41-bcb3d7ea4def, Session Status: ReadyForSetup, Exception: Microsoft.ADF.Contract.AdfHttpResponseException: BadRequest at Microsoft.ADF.DataflowSparkLibrary.DatabricksImpl.ArcadiaDebugRunJob.RunCommandAndGetOutputAsync(String sessionId, String command, CancellationToken cancellationToken, String sourceId) in C:__w\1\s\DataflowManager\src\DataflowManager.Core.Mig\DatabricksImpl\ArcadiaDebugRunJob.cs:line 90 at Microsoft.ADF.DataflowSparkLibrary.DatabricksImpl.ArcadiaDebugRunJob.InitializeDebugSessionAsync(String sessionId, String code, CancellationToken cancellationToken) in C:__w\1\s\DataflowManager\src\DataflowManager.Core.Mig\DatabricksImpl\ArcadiaDebugRunJob.cs:line 63 at Microsoft.ADF.DataflowSparkLibrary.DebugRunJob.SetupDebugSession(SparkInfo sparkInfo, String sessionId, SessionInfo sessionInfo, SystemProperties systemProperties, CancellationToken cancellationToken) in C:__w\1\s\DataflowManager\src\DataflowManager.Core.Mig\DatabricksImpl\DebugRunJob.cs:line 402 at Microsoft.ADF.DataflowManager.DebugRunHandler.CheckoutComputeFromDebugPool(ExecuteDebugRunRequest debugRequest, DataflowComputeProperties dataflowComputeProperties, SystemProperties systemProperties, CancellationToken cancellationToken) in C:__w\1\s\DataflowManager\src\DataflowManager.Core.Mig\JobHandlers\DebugRunHandler.cs:line 1789
Azure Data Factory
{count} votes
-
PRADEEPCHEEKATLA 90,641 Reputation points Moderator2023-06-14T09:51:45.7966667+00:00 @Kyle West - Thanks for the question and using MS Q&A platform.
In order to investigate this issue further, could you please check and verfiy the below steps you can take to resolve the issue:
Ensure that you have the necessary permissions to create and run dataflows in Azure Data Factory. You need to be a member of the
Data Factory ContributororData Factory Ownerrole to create and run dataflows.Select ADF => Access Control(IAM) => View my access
While enabling the Data flow debug are you able to see this pop-up as shown below:
After enabling data flow debug, are you experiencing any error message? If yes, please do share the complete stack trace of the error message?
You can use the monitoring view for debug sessions above to view and manage debug sessions.
And also share the ADF name and region?
Looking forward for your response.
-
Kyle West 0 Reputation points
2023-06-14T18:30:31.85+00:00 To answer your first question - I am an 'owner' under the Access Control (IAM) settings.
Regarding the pop-up, toggling my debug switch shows that pop-up with no issues.
I have an error when I toggle the dataflow debug switch on & it's generating my spark cluster. Here is the error.
Creating debug session failed: Failed to setup debug session while checking out from debug pool. Session Id: 302af5a4-db6d-4b01-865d-75c07acf780b, Session Status: ReadyForSetup, Exception: Microsoft.ADF.Contract.AdfHttpResponseException: BadRequest
at Microsoft.ADF.DataflowSparkLibrary.DatabricksImpl.ArcadiaDebugRunJob.RunCommandAndGetOutputAsync(String sessionId, String command, CancellationToken cancellationToken, String sourceId) in C:__w\1\s\DataflowManager\src\DataflowManager.Core.Mig\DatabricksImpl\ArcadiaDebugRunJob.cs:line 88
at Microsoft.ADF.DataflowSparkLibrary.DatabricksImpl.ArcadiaDebugRunJob.InitializeDebugSessionAsync(String sessionId, String code, CancellationToken cancellationToken) in C:__w\1\s\DataflowManager\src\DataflowManager.Core.Mig\DatabricksImpl\ArcadiaDebugRunJob.cs:line 57
at Microsoft.ADF.DataflowSparkLibrary.DebugRunJob.SetupDebugSession(SparkInfo sparkInfo, String sessionId, SessionInfo sessionInfo, SystemProperties systemProperties, CancellationToken cancellationToken) in C:__w\1\s\DataflowManager\src\DataflowManager.Core.Mig\DatabricksImpl\DebugRunJob.cs:line 399
at Microsoft.ADF.DataflowManager.DebugRunHandler.CheckoutComputeFromDebugPool(ExecuteDebugRunRequest debugRequest, DataflowComputeProperties dataflowComputeProperties, SystemProperties systemProperties, CancellationToken cancellationToken) in C:__w\1\s\DataflowManager\src\DataflowManager.Core.Mig\JobHandlers\DebugRunHandler.cs:line 1718
Here is my ADF name & Region
Impact-datafactory
West US 3
-
PRADEEPCHEEKATLA 90,641 Reputation points Moderator
2023-06-19T04:57:52.9833333+00:00 @Kyle West - Thanks for sharing the detailed error message.
I agree that this issue looks strange and I wasn't able to reproduce this issue. If you have a support plan could you please file a support ticket for deeper investigation and do share the SR# with us? In case if you don't have a support plan please let us know here.
-
Kyle West 0 Reputation points
2023-06-20T20:56:38.2166667+00:00 I dont think I have a support plan. I don't even know what that is or where to support a ticket.
Shouldn't the error code I provided tell you why the debugger won't turn on?
-
PRADEEPCHEEKATLA 90,641 Reputation points Moderator
2023-06-22T05:52:01.6933333+00:00 @Kyle West - In your case, this issue cannot be solved on the public forum and need to open a support ticket in order to investigate and troubleshoot your issue.
This article - Create an Azure support request helps to create a support ticket in Azure Portal.
Hope this helps.
-
PRADEEPCHEEKATLA 90,641 Reputation points Moderator
2023-06-29T08:48:40.9333333+00:00 @Kyle West - Did you get a chance to open a support ticket?
-
Kyle West 0 Reputation points
2023-07-07T15:52:48.6133333+00:00 No. I don't know how to open a support ticket.
Instead, I have just been trying to figure it out on my own. Surely there is a reason my dataflow debugger runs into an error.
Sign in to comment
Sign in to answer