DDL opertaions against Azure Cosmos DB Cassandra API from Spark

Question

Hi,

I'm having very strange behavior on DDL operations against Cosmos DB w/ Cassandra API from Spark running in Databricks.

When creating a keyspace and table as the following, it does execute the create statements with no errors, BUT it really didn't create a keyspace and table in the Cosmos DB.

  cdbConnector.withSessionDo{session =>
      session.execute(s"CREATE KEYSPACE IF NOT EXISTS ${keyspace} WITH replication = {'class': 'SimpleStrategy', 'replication_factor': 1}")
      session.execute(s"CREATE TABLE IF NOT EXISTS ${keyspace}.${tablename}(place_id TEXT, pois TEXT, PRIMARY KEY(place_id))")
      session.execute(s"SELECT * FROM ${keyspace}.${tablename} LIMIT 1")
  }

Eventually, the last select statement fails complaining the keyspace/table doesn't exist.

One thing to notice is that if I run the above DDL operations multiple times over and over again, then it occasionally succeeds to create the keyspace and table. Hence, the behavior is somehow random that I can't find any pattern or cause of the issue. Also, note that I already tried to separate out these statements into multiple withSessionDo, but I see same behavior.