spark.read not working on Synapse to Cosmos link

Question

The code below worked always. However today I tried it with a Container Name of length 34 characters.

df = spark.read\
.format("cosmos.olap")\
.option("spark.synapse.linkedService", <Linked Service Name>)\
.option("spark.cosmos.container", <Container Name>)\
.load()\

Since then I keep getting this error on executing the code above with any new containers created from that point on -
ParseException:
mismatched input '<EOF>' expecting {'SELECT', 'FROM', 'ADD'..............

I've already tried switching over to a new Database, a new Cosmos SQL account, a new Synapse workspace, but the error doesn't go away for new containers. Containers created before trying out the 34 character length container name, continue to work fine.

Answer 1

Hello @Sumit Kumar ,

Thanks for the question and using MS Q&A platform.

We haven't see this kind of the behaviour before, as per the repro I had tried with a container name of length 34 & 45 characters and able to run without any issues.

You can try to below steps to connect Azure Cosmos DB using Apache Spark 2 in Azure Synapse Link:

Step1: Go to Linked and select the Azure Cosmos DB linked service and test the connection.

Step2: Select the container and click on actions => New Notebook => Load to DataFrame.

Step3: Select the Spark pool and run the code to load the dataframe from container name of length 34.

Step4: Select the Spark pool and run the code to load the dataframe from container name of length 45.

For more details, refer to Interact with Azure Cosmos DB using Apache Spark 2 in Azure Synapse Link.

After trying the above steps, if you still get the error message, please do let us know.

Hope this will help. Please let us know if any further queries.

------------------------------

• Please don't forget to click on or upvote button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how
• Want a reminder to come back and check responses? Here is how to subscribe to a notification
• If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators

Answer 2

I get that error for any container which is empty. Once it has items, it works.