SaveTable in Datalake Table of Azure Synapse from Synapse Spark pool

Question

I have create a new lake database called DeltaTables and when I'm trying to write a spark dataframe into that Datalake it's showing me success message but I don't see any table created. where as If I write same dataframe in default lakedatabase which is by default came up in Synapse I can see the Table created. So just need some assistance on how to create a lakedatabase that is similar to default lakedatabase which Synapse has created

The code which I have used is df.write.mode("overwrite).saveAsTable("DeltaTables.Test")

The other difference I see with compared to default lakedatabase is default one allows to create views as well. Attached a snap for referrence

Answer 1

Hello @Mohammed Sohail ,

Thanks for the question and using MS Q&A platform.

As per the repro, we are able to write a spark dataframe into lake database and able to see the table created.

Even we can query the table which has created.

If you try to create a view on a lake database, you will be getting the below error as this is not supported.

Appreciate if you could share the feedback on our feedback channel. Which would be open for the user community to upvote & comment on. This allows our product teams to effectively prioritize your request against our existing feature backlog and gives insight into the potential impact of implementing the suggested feature.

Hope this will help. Please let us know if any further queries.

------------------------------

• Please don't forget to click on or upvote button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how
• Want a reminder to come back and check responses? Here is how to subscribe to a notification
• If you are interested in joining the VM program and help shape the future of Q&A: Here is jhow you can be part of Q&A Volunteer Moderators

Answer 2

Thanks @PRADEEPCHEEKATLA-MSFT , Actually the tables are getting created in lake database but I was unable to see, but when I connected to powerBI it's listing me out all the tables. Attached some screen shots for your referrence

This is from PowerBI

The below is from Synapse

when you compare those images, there are more Tables showing up in PowerBI which I too don't know that it got created untill I connected to PowerBI. Please let me know If you have any idea regarding the same.

and Talking about views if you try creating views from Synapse notebook using df.createOrReplaceTempView("<view-name>") it's getting created I think this is creating only in memory but I was just wondering why Synapse has given an option of Views on lakedatabse. If you can clarify on that It would be great

Thanks much !!