Microsoft.EventGrid Registration ISSUE in my subscription

Question

Hi Team,

I am doing a POC in my organisation to bring the incremental in azure blob storage into delta table.

I am using databricks autoloader to fetch the data from Azure data-lake-gen2 container to data-bricks incrementally.

Though I have registered for Microsoft.EventGrid in my trail subscription. I am getting the following error, when reading the data

java.lang.RuntimeException: Failed to create event grid subscription. Please ensure that Microsoft.EventGrid is
registered as resource provider in your subscription.

Microsoft.EventGrid is registered as resource provider in my subscription. I am attaching screenshot as an evidence

Following is the kinda code in databricks notebook:

=================================

cloudfile={
"cloudFiles.subscriptionId" : sub_id,
"cloudFiles.connectionString" : queuesas,
"cloudFiles.format": "csv",
"cloudFiles.tenantId" : tenant_id,
"cloudFiles.clientId" : client_id,
"cloudFiles.clientSecret" : secret_value,
"cloudFiles.resourceGroup" : rg,
"cloudFiles.useNotifications" : "true"
}

df = (spark \
.readStream \
.format("cloudFiles") \
.options(**cloudfile) \
.schema(schema) \
.load(f"wasbs://{container_name}@{storage_account_name}.blob.core.windows.net/")
)

(df.writeStream \
.format("delta") \
.option("checkpointLocation","/FileStore/tables/stream_checkpoint") \
.start("companywiseAnalysis")
)

I cannot understand the issue here. could you help me here

Answer 1

I was seeing a similar issue with our ADLS gen2 account where the data was landing. The solution for us was to enable the Allow Azure services on the trusted services list to access this storage account setting on the storage account. The account was pretty closed off with the Enabled from selected virtual networks and IP addresses setting enabled to specified VNets and Subnets.