Getting below error when trying to create a Data Quality Monitor for the table.""Cannot create Monitor because it exceeds the number of limit 500."

Question

Getting below error when trying to create a Data Quality Monitor for the table." "Cannot create Monitor because it exceeds the number of limit 500."

Answer 1

Hi @Tanima Saxena

Thank you for contacting to Microsoft QA.

It looks like you're running into a quota issue while trying to create a Data Quality Monitor for a table. The error message "Cannot create Monitor because it exceeds the number of limit 500" suggests that you're hitting a resource quota in Unity Catalog.

Here's what you can try:

Check Quota Usage: Make sure to check how many monitors you currently have. Unity Catalog limits the number of data quality monitors (100 monitors per metastore). If you're near this limit, you'll need to delete some unused monitors to free up space.

Delete Unused Resources: If you have any old or unused monitors, consider deleting them to stay under the quota limits. You can do this through the Unity Catalog UI or API.

Request a Quota Increase: If you need more than the current limits, you'll want to contact your Azure Databricks account team to request a quota increase, as creating a support case for this won't be effective.

Monitor Quota Usage: Utilize the Unity Catalog resource quotas APIs to keep track of your quota usage. This might help you manage resources better moving forward.

For more detailed information, you can check the following resources:

• Unity Catalog Resource Quotas
• Monitor your usage of Unity Catalog resource quotas

Hope this helps resolve your issue! If you're still facing problems, let me know and we can dig deeper!

Follow-Up Questions:

1. How many monitors do you currently have set up in Unity Catalog?
2. Have you already tried deleting any unused monitors or resources?
3. Are you operating under any specific quotas that we should be aware of?
4. Could you provide the specific context of how you're trying to create the monitor (e.g., via UI, API, CLI)?