question

tdppnr avatar image
1 Vote"
tdppnr asked tdppnr edited

Databricks Notebook Activity parameter problem

10333-parameter.jpg



I feel this is a bug but not sure if it is with ADF or Databricks. I am running a notebook using ADF notebook activity. My notebook has a widget for which I pass the value from ADF. As I need to manually enter the parameter name while configuring the notebook activity, I accidentally added an extra space. When I ran the pipeline it succeeded but not the expected output from notebook. It appears that due the extra space, the value is not passed to notebook but at the same time I would expect the pipeline to fail with some message " Parameter with extra space not found"

azure-data-factoryazure-databricks
parameter.jpg (15.2 KiB)
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

1 Answer

HimanshuSinha-MSFT avatar image
0 Votes"
HimanshuSinha-MSFT answered tdppnr edited

<p>Hello @tdppnr, <br />Welcome to the Q&A . </p>
<p>To make sure that I understand the issue better i went ahead and tried to find out how does ADB handles spaces with the widgets . I found that the it just removed that . Please check the snapshot below . </p>
<p><img src="/answers/storage/attachments/10442-1.gif" alt="10442-1.gif" /></p>
<br /><p>Went ahead and initated the notebook from the ADF now . Passing the parameter with the spaces . </p>
<p><img src="/answers/storage/attachments/10451-adf-ui.gif" alt="10451-adf-ui.gif" /> </p>
<br /><p>Yes having spaces does not makes the ADF fails , but then I think adding the below expression on the ADF side , will help .</p>
<pre><code> @trim(' HImanshu ')
</code></pre>
<br /><br /><br /><br /><br /><p>Thanks & stay safe </p>
<p>Himanshu<br /></p>
<hr />
<p>Please do consider to click on <strong>"Accept Answer"</strong> and <strong>"Up-vote"</strong> on the post that helps you, as it can be beneficial to other community members</p>
<br /><br />


1.gif (175.8 KiB)
adf-ui.gif (558.1 KiB)
· 1
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hi @HimanshuSinha-5732,


Thank you for the reply. I dont see how we can TRIM() the variable name on the left side in ADF or "dbutils.widgets.get(@trim('variable'))" in databricks. I wanted to understand if something can be done at product level (Either ADF or Databricks). I spent considerable amount of time investigating this and was lucky to find the issue. Either fixing or adding a note in official documentation will help people like me


0 Votes 0 ·