Serverless SQL pool query returns NULL for column values in COSMOS DB documents

Sushanta Swain 1 Reputation point
2022-11-18T10:41:35.667+00:00

Hi Team,

I have enabled analytical store for my cosmos DB container and trying to create views for the tables in Serverless SQL pool. When I query via synapse, It returns NULL for some columns.

COSMSOS DB document structure:

{
"id": "inRiver_1130030",
"type": "Resource",
"_partition": "test",
"_pimdate": "11/11/2022 02:21:02 PM",
"ResourceActive": true,
"ResourceFileId": 459303,
"ResourceFilename": "FIeldMaps_datamart.xml",
"ResourceMimeType": "text/xml",
"ResourceName": "FIeldMaps_datamart.xml",
"ResourceType": "Configuration",
"_rid": "ZZVmAL1xDN1hAAAAAAAAAA==",
"_self": "dbs/ZZVmAA==/colls/ZZVmAL1xDN0=/docs/ZZVmAL1xDN1hAAAAAAAAAA==/",
"_etag": "\"df00790d-0000-0300-0000-636e5a4e0000\"",
"_attachments": "attachments/",
"_ts": 1668176462
}

261874-image.png

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,364 questions

1 answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. BhargavaGunnam-MSFT 26,136 Reputation points Microsoft Employee
    2022-11-22T05:05:17.047+00:00

    Hello @Sushanta Swain ,
    When I query from SQL and PySpark, I am not seeing any null values.
    A similar thread has been discussed here.

    Here is the root cause of the issue:
    If you rename a property, in one or many documents, it will be considered a new column. If you execute the same rename in all documents in the collection, all data will be migrated to the new column and the old column will be represented with NULL values.

    For more details, refer to What is Azure Cosmos DB analytical store

    I hope this helps. Please let us know if you have any further questions.

    262800-image.png

    262913-image.png

    ------------------------------

    • Please don't forget to click on 130616-image.png and upvote 130671-image.png button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how
    • Want a reminder to come back and check responses? Here is how to subscribe to a notification
    • If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators