Refresh issue with Cosmos dB connection

Question

Hi,

I noticed one strange thing with my power bi report as I made a connection cosmos dB (used a query) and get the data applied some transformation created visual and published on the service now refresh is schedule for every day. Now some records get deleted from cosmos dB and have some new addition of the records but my report is not reflecting that changes even when I try to do manual refresh in desktop report still not refreshing so every time I have to choose as new query and make a new connection and do all the same transformation then it's getting updated data so its rework again and again. Connector: Azure Cosmos DB V1. can anyone help with this?

---

Moved from: Community Center | Discuss the Q&A site | Get started on Q&A

Answer 1

Hi @Khusbu Kumari,
Welcome to the Microsoft Q&A and thank you for posting your questions here.
Your refresh misses changes because folding breaks with the V1 connector and Import can’t see hard deletes. Move to Cosmos DB V2, keep early steps fold‑safe, enable Incremental Refresh on a time column, and handle deletes via soft delete or Change Feed. This removes the need to “recreate the query” and makes daily/Manual refresh reflect inserts and deletes reliably.

Query folding breaks early in Power Query (after certain transforms), so refresh reuses a stale snapshot and doesn’t re‑hit Cosmos properly.

Import models don’t naturally detect hard deletes in Cosmos; without tombstones or a foldable filter, removed rows remain until a full reload.

The Cosmos DB V1 connector has known quirks with folding and cross‑partition behavior; these are improved in V2.

What to do

1. Switch to the Azure Cosmos DB (SQL API) V2 connector Reconnect with V2, point to the same database/collection, and keep early steps fold‑friendly (Source → Filter rows → Select columns → simple type changes).
2. Preserve query folding In Power Query, right‑click the last step → “View Native Query.” If it’s disabled, move heavy transforms (merges, custom columns, row‑by‑row functions) after import or implement them as DAX measures. Split into:
   • Staging query (folding to Cosmos)
   • Transform query (non‑folding, post‑import)
3. Configure Incremental Refresh (so inserts/updates flow without full reloads)
   • Add RangeStart and RangeEnd parameters.
   • Filter the query on a reliable time field that folds (Cosmos _ts or your updatedAt).
   • Set table Incremental Refresh policy (e.g., store 3 years, refresh last 7 days).
4. Handle deletes explicitly
   • Soft delete pattern: add isDeleted (or status) to documents; filter isDeleted = false in the foldable portion of the query.
   • Hard delete pattern: use Cosmos Change Feed to capture deletes (tombstones) into a staging store (Azure Storage/Azure SQL) and build the model from there; schedule that pipeline + dataset refresh. For occasional cleanups, trigger a Full Refresh in the Service.
5. Refresh hygiene
   • Desktop: File → Options → Data Load → Clear cache, then Refresh.
   • Queries: ensure Include in report refresh is enabled for each query.
   • Service: Cosmos is cloud; avoid binding the dataset to a misconfigured gateway. Check Refresh history for failures.

Answer 2

It sounds like you're experiencing issues with data refresh in your Power BI report connected to Azure Cosmos DB. Here are a few points to consider:

1. Data Refresh Types: In Power BI, there are different types of data refreshes, including scheduled refresh and manual refresh. If records are deleted or added in Cosmos DB, the changes may not reflect in Power BI if the data refresh is not executed properly. Ensure that your scheduled refresh is set up correctly and that you are using the right connector.
2. Manual Refresh: When you perform a manual refresh in Power BI Desktop, it should ideally fetch the latest data from the source. If it’s not working, check if there are any errors in the refresh process that might indicate issues with the connection or the query.
3. Query Caching: Power BI caches query results to improve performance. This means that even if the underlying data changes, you might not see those changes reflected immediately unless a refresh is executed. You may want to configure automatic page refresh in the Power BI service to keep your visuals updated with the latest data changes.
4. Schema Changes: If the schema of your data source changes (e.g., columns are added or removed), it can cause refresh failures. Make sure that your queries and transformations are aligned with the current schema of your Cosmos DB.
5. Reconnection: If you find yourself frequently needing to reconnect and redo transformations, consider saving your transformation steps in Power BI. This way, you can easily reapply them without starting from scratch each time.

If the problem persists, it may be beneficial to review the Power BI documentation on data refresh and troubleshooting incremental refresh issues for more detailed guidance.

---

References:

• Understanding data refresh
• Automatic page refresh in Power BI