Covert Azure synapse notebooks saved in Azure DevOps[in json] to python or jupyter

Question

Covert Azure synapse notebooks saved in Azure DevOps[in json] to python or jupyter

MADHUSUDAN PANWAR 106

Hi,

I have various notebooks saved in Azure Synapse connected to Azure DevOps. They are stored in DevOps as .json file. Is there any easy way to convert notebooks saved in .json file to .py or .ipynb file? I know there is an option in Synapse to export notebook to .py file. However, I have lots of notebooks and cannot do this manual process. Is there any tool or any smart way of doing the same?

Similar question was asked here

User's image

AnnuKumari-MSFT 34,566 Reputation points Microsoft Employee Moderator

2023-07-04T05:26:31.12+00:00

Hi MADHUSUDAN PANWAR ,

Thankyou for using Microsoft Q&A platform and thanks for posting your question here.

As per my understanding, you want to convert notebook json into python files in synapse. Please let me know if that is not the requirement here.

There is no direct way of achieving this requirement , however you can write your own custom code to get this done. Here are few of the resources which can be of your help:

Convert JSON to dictionary in Python

Convert JSON object to dictionary in Python

How to Convert JSON Data Into a Python Object

Hope it helps. Please let us know how it goes. Thankyou
AnnuKumari-MSFT 34,566 Reputation points Microsoft Employee Moderator

2023-07-13T09:58:04.68+00:00

Hi MADHUSUDAN PANWAR ,

Did you get a chance to look at the previous response? Kindly let me know how it went. If you got the workaround, encouraging you to share the solution with the community. Thankyou
Valencia, Fabian 10 Reputation points

2024-01-24T00:22:58.77+00:00

@AnnuKumari-MSFT How is that a good answer. Microsoft Synapse saves the data as Json. But if you want to use the Azure CLI or Azure Synapse Power Shell, it requires a ipynb file. The question if valid, how do we convert from json to ipynb and Microsoft should provide a way to do so.
MADHUSUDAN PANWAR 106 Reputation points

2024-01-25T14:14:08.34+00:00

Hi @AnnuKumari-MSFT I still haven't found the answer. It would be great if you can share something to convert .json notebook to .ipynb file

Valencia, Fabian 10

I gave up on the ipynb file. I basically resorted to use the rest API as documented here:

https://learn.microsoft.com/en-us/rest/api/synapse/data-plane/notebook/create-or-update-notebook?view=rest-synapse-data-plane-2020-12-01&viewFallbackFrom=rest-synapse-data-plane-2021-04-01&tabs=HTTP

This is the code that works for me to upload via powershell

$content = ConvertFrom-Json( [string](Get-Content $JsonFile))
        $notebookPath = $JsonFile
        $notebookName = $($content.name)


        #$token = (Get-AzAccessToken -Resource "https://dev.azuresynapse.net/").Token

        $ApiVersion = "2021-04-01"
        $Endpoint = "https://{0}.dev.azuresynapse.net/notebooks/{1}?api-version={2}" -f $Workspace, $NotebookName, $ApiVersion
        $Headers = @{
            'accept'='application/json';    
            'Authorization' = "Bearer $token"
        } 
    
        $content = Get-Content -Path $notebookPath -Raw
        $SynapseLSResponse = Invoke-WebRequest -Method Put -Body $content -ContentType "application/json" -Uri $Endpoint -Headers $Headers

Hope it helps!

Your answer

AnnuKumari-MSFT 34,566 Reputation points Microsoft Employee Moderator

2023-07-04T05:26:31.12+00:00

Hi MADHUSUDAN PANWAR ,

Thankyou for using Microsoft Q&A platform and thanks for posting your question here.

As per my understanding, you want to convert notebook json into python files in synapse. Please let me know if that is not the requirement here.

There is no direct way of achieving this requirement , however you can write your own custom code to get this done. Here are few of the resources which can be of your help:

Convert JSON to dictionary in Python

Convert JSON object to dictionary in Python

How to Convert JSON Data Into a Python Object

Hope it helps. Please let us know how it goes. Thankyou
AnnuKumari-MSFT 34,566 Reputation points Microsoft Employee Moderator

2023-07-13T09:58:04.68+00:00

Hi MADHUSUDAN PANWAR ,

Did you get a chance to look at the previous response? Kindly let me know how it went. If you got the workaround, encouraging you to share the solution with the community. Thankyou
Valencia, Fabian 10 Reputation points

2024-01-24T00:22:58.77+00:00

@AnnuKumari-MSFT How is that a good answer. Microsoft Synapse saves the data as Json. But if you want to use the Azure CLI or Azure Synapse Power Shell, it requires a ipynb file. The question if valid, how do we convert from json to ipynb and Microsoft should provide a way to do so.
MADHUSUDAN PANWAR 106 Reputation points

2024-01-25T14:14:08.34+00:00

Hi @AnnuKumari-MSFT I still haven't found the answer. It would be great if you can share something to convert .json notebook to .ipynb file
Valencia, Fabian 10 Reputation points

2024-01-25T19:34:57.27+00:00

I gave up on the ipynb file. I basically resorted to use the rest API as documented here:

https://learn.microsoft.com/en-us/rest/api/synapse/data-plane/notebook/create-or-update-notebook?view=rest-synapse-data-plane-2020-12-01&viewFallbackFrom=rest-synapse-data-plane-2021-04-01&tabs=HTTP

This is the code that works for me to upload via powershell

$content = ConvertFrom-Json( [string](Get-Content $JsonFile)) $notebookPath = $JsonFile $notebookName = $($content.name) #$token = (Get-AzAccessToken -Resource "https://dev.azuresynapse.net/").Token $ApiVersion = "2021-04-01" $Endpoint = "https://{0}.dev.azuresynapse.net/notebooks/{1}?api-version={2}" -f $Workspace, $NotebookName, $ApiVersion $Headers = @{ 'accept'='application/json'; 'Authorization' = "Bearer $token" } $content = Get-Content -Path $notebookPath -Raw $SynapseLSResponse = Invoke-WebRequest -Method Put -Body $content -ContentType "application/json" -Uri $Endpoint -Headers $Headers

Hope it helps!

Share via

Covert Azure synapse notebooks saved in Azure DevOps[in json] to python or jupyter

Your answer