Azure Data Lake Storage
An Azure service that provides an enterprise-wide hyper-scale repository for big data analytic workloads and is integrated with Azure Blob Storage.
1,366 questions
0 answers
I can't solve the python error, so please let me know.
I am using pyhon in synapse notebook. I'm trying to convert each of the three sheets in one macro-enabled excel file to a csv file. However, I have also granted contributor and blob permissions, but it doesn't work. I would like to know how…
asked 2022-08-02T08:45:01.877+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(198.4, 30%, 29%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EN9%3C/text%3E%3C/svg%3E)
NishimuraChinatsu-9854
756
Reputation points
commented 2022-08-15T16:26:32.587+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(192, 26%, 28%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EHM%3C/text%3E%3C/svg%3E)
HimanshuSinha-msft
19,381
Reputation points Microsoft Employee
0 answers
How to avoid creation of Snapshot folder during SynapseLink's data lake exports
Hello all, We are using Synapse Link to export Dynamics CRM data to ADLS Gen2. The CRM entities are exported in CSV format in CDM folders. They don't have headers. The records are splitted into multiple CSV files. We are trying to reference these…
asked 2022-07-19T18:38:13.943+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(281.6, 7.000000000000001%, 37%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMB%3C/text%3E%3C/svg%3E)
Mehmet BAŞERDEM
41
Reputation points
commented 2022-08-15T16:20:14.3+00:00
HimanshuSinha-msft
19,381
Reputation points Microsoft Employee
3 answers
Connecting to MongoDB 4.4.1
Hi, We are using Synapse Pipelines in order to connect a MongoDB (version: 4.4.1). According to the documentation, the MongoDB connector supports versions up to 4.2.…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(41.6, 97%, 14%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EM%3C/text%3E%3C/svg%3E)
2 answers
Azure synapse multiple workspace same sql pool
Hi , I have a requirement to created synapse analytics workspace for DEV, QA and Prod environment, dev and QA are under one subscription and prod is in another subscription. What is the best approach to setup all the 3 environments keeping costing IN…
asked 2022-08-11T16:56:46.943+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(160, 68%, 25%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ER%3C/text%3E%3C/svg%3E)
Ram001
6
Reputation points
answered 2022-08-15T05:38:43.75+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(70.4, 96%, 16%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ECG%3C/text%3E%3C/svg%3E)
Christopher Geckert
91
Reputation points
2 answers
How to pass count of number rows affected in dataflow sink to piepleine
I have created one dataflow where source is Azure SQL and sink is inline database Delta table. I want to get number of rows inserted in sink and pass those counts to pipeline and assign it to variable.
asked 2022-08-09T11:11:31.093+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(297.6, 52%, 38%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EHD%3C/text%3E%3C/svg%3E)
heta desai
247
Reputation points
commented 2022-08-13T08:42:38.747+00:00
AnnuKumari-MSFT
31,716
Reputation points Microsoft Employee
0 answers
Azure Synapse Merge Command is failing with exception - "Updating a distribution key column in a MERGE statement is not supported."
I am trying to merge a target table with a source table based on some conditions. It is supposed to insert the records from the source table to the target table. Though it won't update the distribution key, it keeps failing with the exception,…
asked 2022-08-04T13:15:02.19+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(134.4, 8%, 23%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERG%3C/text%3E%3C/svg%3E)
Rahul Glenn Rajkumar
1
Reputation point
commented 2022-08-12T07:33:49.59+00:00
Rahul Glenn Rajkumar
1
Reputation point
2 answers
Why synapse analytics
Hi friends what is the max storege limits of azure synapse analytics how can project decide to use synapse which is evaluation process
asked 2022-07-19T12:12:56.643+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(166.4, 15%, 26%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EA%3C/text%3E%3C/svg%3E)
Anshal
1,966
Reputation points
commented 2022-08-12T06:31:37.417+00:00
AnnuKumari-MSFT
31,716
Reputation points Microsoft Employee
1 answer
Facing 'DFExecutorUserError' when using mapping data flow for the lake database
I'm trying to map the data from my data lake (workspace default storage) to the table of the lake database in Synapse. However, whenever I tried to execute the auto-generated data flow, it gives me the error as below: Operation on target…
asked 2022-08-09T13:17:08.44+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(220.8, 24%, 31%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EF%3C/text%3E%3C/svg%3E)
Feiyi
1
Reputation point
commented 2022-08-12T06:25:20.543+00:00
ShaikMaheer-MSFT
38,126
Reputation points Microsoft Employee
1 answer
I want to use python in syanpsenotebook activity, is there any difference in what I can do with normal python?
Is it possible to send an email using python in synapse notebook activity? Also, is there anything I can't do with python in synapse's notebook activity?
asked 2022-08-04T06:37:49.173+00:00
NishimuraChinatsu-9854
756
Reputation points
commented 2022-08-12T05:10:10.1+00:00
ShaikMaheer-MSFT
38,126
Reputation points Microsoft Employee
1 answer
One of the answers was accepted by the question author.
How to execute query on Microsoft Service Now environment using Python using REST API
Hello Team, We need Python API documentation to fetch data from Microsoft Service Now. We have ADX type query like u_incident, sys_created_by which should be execute on the environment and should get data like incidence, details. We are now…
asked 2022-08-01T11:26:07.877+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(172.8, 6%, 26%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EPR%3C/text%3E%3C/svg%3E)
Prashant Rewatkar
96
Reputation points
accepted 2022-08-11T16:46:51.993+00:00
Prashant Rewatkar
96
Reputation points
2 answers
One of the answers was accepted by the question author.
Load data into synapse delta lake from Azure SQL database using synapse pipeline
I want to load data from azure sql database to delta lake table using synapse pipeline. I tried using copy activity but there is no target type available to load data into delta lake directly. So, using copy activity I am reading data from azure sql and…
asked 2022-07-11T12:36:31.323+00:00
heta desai
247
Reputation points
accepted 2022-08-11T16:22:31.517+00:00
heta desai
247
Reputation points
1 answer
One of the answers was accepted by the question author.
queryPipelineRuns REST query for Azure Synapse workspaces doesn't work with filters
Following the documentation here: https://learn.microsoft.com/en-us/rest/api/synapse/data-plane/pipeline-run/query-pipeline-runs-by-workspace Adding a request body doesn't appear to work. Running no request body: Then running with…
asked 2022-08-09T14:36:38.237+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(48, 78%, 14%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ETB%3C/text%3E%3C/svg%3E)
Thomas Bailey
36
Reputation points
commented 2022-08-11T09:08:19.427+00:00
Thomas Bailey
36
Reputation points
1 answer
How to combine some files automatically in Azure Synapse Analytics.
I am working as a data engineer and I have to combine some files into one file every day. Here is what I would like to do: Upload file(.gz) to Azure Blob Storage every day. Unzip the file to parquet format. Combine some files into one…
asked 2022-06-22T09:03:36.28+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(19.2, 80%, 11%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EKS%3C/text%3E%3C/svg%3E)
Kakehi Shunya (筧 隼弥)
201
Reputation points
commented 2022-08-11T04:47:30.887+00:00
AnnuKumari-MSFT
31,716
Reputation points Microsoft Employee
1 answer
How to use common variable to store the failure status of one activity out of two activies
In Pipeline there are two activities for which I want to manage the success/failure status. There is copy activity if it fails the Status variable value will set to Failure. It copy activity is success that next activity notebook will execute and if…
asked 2022-06-08T17:15:05.85+00:00
heta desai
247
Reputation points
commented 2022-08-11T04:45:09.35+00:00
AnnuKumari-MSFT
31,716
Reputation points Microsoft Employee
3 answers
Synapse CopyInto with ErrorFolder Operating system error code 5 (Access is denied)
I'm trying to use "Copy Into" to bulk load data into Synapse from a CSV file. Here is the command I used. COPY INTO dbo.[StageProducts2] FROM 'https://<<mystorageaccount>>.blob.core.windows.net/data-files/Instacart/products.csv' …
asked 2020-10-15T11:40:53.08+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(147.20000000000002, 40%, 24%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ESU%3C/text%3E%3C/svg%3E)
satish umapathy
26
Reputation points
answered 2022-08-09T18:29:44.817+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(246.4, 22%, 33%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EPK%3C/text%3E%3C/svg%3E)
Prathyusha Kanala
1
Reputation point
1 answer
Data Flow - Window Transformation - NTILE Expression
I'm attempting to assign quartiles to a numeric source data range as it transits a data flow. I gather that this can be accomplished by using the ntile expression within a window transform. I'm failing in my attempt to use the documentation…
asked 2022-08-03T15:03:37.073+00:00
Wilson, Chris
6
Reputation points
commented 2022-08-09T15:46:47.207+00:00
HimanshuSinha-msft
19,381
Reputation points Microsoft Employee
2 answers
Unable scale synapse pool using ADF managed identity
Hi Folks, We are trying to create pipeline to scale up synapse pool dynamically using below documentation: https://microsoft-bitools.blogspot.com/2021/01/scaling-azure-dedicated-sql-pools-from.html Datafactory account has Sql contributor role…
asked 2022-07-18T14:24:22.89+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(179.20000000000002, 68%, 27%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERR%3C/text%3E%3C/svg%3E)
Ritu R Kumari
1
Reputation point
commented 2022-08-09T12:30:25.477+00:00
AnnuKumari-MSFT
31,716
Reputation points Microsoft Employee
1 answer
How to do I connect to dataverse using pyspark/python in synapse notebook
How to do I connect to dataverse using pyspark/python in synapse notebook. We tried to link from dataverse but the dataverse and datalake are in different locations. What is the other solution to connect to dataverse
asked 2022-08-04T08:35:31.36+00:00
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(284.8, 19%, 37%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERP%3C/text%3E%3C/svg%3E)
Radha Pusarapu (Ernst & Young LLP)
36
Reputation points Microsoft Vendor
commented 2022-08-09T07:03:30.817+00:00
Radha Pusarapu (Ernst & Young LLP)
36
Reputation points Microsoft Vendor
2 answers
I would like to know the python code that converts Excel data into a csv file for each sheet.
I would like to know the pyhon code that converts Excel data with 3 sheets into a csv file with synapse notebook acyivity. The image looks like the one below. Must be pyhon because it is a macro-enabled Excel file. ]1]1
asked 2022-07-29T09:53:29.513+00:00
NishimuraChinatsu-9854
756
Reputation points
commented 2022-08-09T06:05:20.137+00:00
PRADEEPCHEEKATLA-MSFT
79,376
Reputation points Microsoft Employee
0 answers
Can not read data from cosmos db analytical store via managed Virtual Network in synapse workspace.
i created synapse workspace with managed Virtual Network .Then add Managed private endpoints to cosmos db both analytical store and sql store . Approved the Managed private. Created the synapse link to cosmos db via managed Vnet runtime.Every thing is…
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(12.8, 70%, 11%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EA%3C/text%3E%3C/svg%3E)