Cannot Create a Synapse Serverless View for DELTA if the delta log does not exist.

Question

You can create Synapse Serverless View for CSV even if the data does not exist. However, you cannot create a Synapse Serverless View for DELTA if the delta log does not exist.

This causes issues when deploying databases to clean environments where the data does not exist.

You should be able to create the view, and if you query the view and the delta log is not there, it should throw an error.

Any suggestions on how to get around this or force create the view?

Answer 1

Hi Toby Riley ,

Welcome to Microsoft Q&A platform and thanks for posting your question here.

I understand that you are facing an issue while creating a Synapse Serverless View for DELTA if the delta log does not exist. You mentioned that you are able to create a Synapse Serverless View for CSV even if the data does not exist. However, you cannot create a Synapse Serverless View for DELTA if the delta log does not exist. Please let me know if my understanding about your query is incorrect.

To address this issue, you can use the following approach:

When you create a Synapse Serverless View for CSV, it does not require any metadata or schema information to be present in the file. However, when you create a Synapse Serverless View for DELTA, it requires the delta log to be present to read the schema information.

If the delta log does not exist, you can create an empty delta log using the delta command-line tool. The delta tool is a command-line interface for working with Delta Lake tables and files. You can use the delta init command to create an empty delta log for a table.

1.Kindly try the following command to create an empty delta log for a table:

delta init <table-path>

Replace <table-path> with the path to the table you want to create an empty delta log for.

2. Once you have created the empty delta log, you can create the Synapse Serverless View for DELTA.
3. Additionally, you can use Partitioned views and result set caching to improve query performance.

Hope it helps. Thankyou

Answer 2

Hi Toby Riley ,

Welcome to Microsoft Q&A platform and thanks for posting your question here.

I understand that you are facing an issue while creating a Synapse Serverless View for DELTA if the delta log does not exist. You mentioned that you are able to create a Synapse Serverless View for CSV even if the data does not exist. However, you cannot create a Synapse Serverless View for DELTA if the delta log does not exist. Please let me know if my understanding about your query is incorrect.

To address this issue, you can use the following approach:

When you create a Synapse Serverless View for CSV, it does not require any metadata or schema information to be present in the file. However, when you create a Synapse Serverless View for DELTA, it requires the delta log to be present to read the schema information.

If the delta log does not exist, you can create an empty delta log using the delta command-line tool. The delta tool is a command-line interface for working with Delta Lake tables and files. You can use the delta init command to create an empty delta log for a table.

1.Kindly try the following command to create an empty delta log for a table:

Copy

delta init <table-path>

Replace <table-path> with the path to the table you want to create an empty delta log for.

1. Once you have created the empty delta log, you can create the Synapse Serverless View for DELTA.
2. Additionally, you can use Partitioned views and result set caching to improve query performance.

Hope it helps. Thankyou