Microsoft.EventGrid Registration ISSUE in my subscription

ARUN P 1 Reputation point
2022-07-01T14:32:12.483+00:00

Hi Team,

I am doing a POC in my organisation to bring the incremental in azure blob storage into delta table.

I am using databricks autoloader to fetch the data from Azure data-lake-gen2 container to data-bricks incrementally.

Though I have registered for Microsoft.EventGrid in my trail subscription. I am getting the following error, when reading the data
216976-screenshot-from-2022-07-01-19-54-47.png
java.lang.RuntimeException: Failed to create event grid subscription. Please ensure that Microsoft.EventGrid is
registered as resource provider in your subscription.

Microsoft.EventGrid is registered as resource provider in my subscription. I am attaching screenshot 217022-screenshot-from-2022-07-01-19-54-47.png as an evidence

Following is the kinda code in databricks notebook:

=================================

cloudfile={
"cloudFiles.subscriptionId" : sub_id,
"cloudFiles.connectionString" : queuesas,
"cloudFiles.format": "csv",
"cloudFiles.tenantId" : tenant_id,
"cloudFiles.clientId" : client_id,
"cloudFiles.clientSecret" : secret_value,
"cloudFiles.resourceGroup" : rg,
"cloudFiles.useNotifications" : "true"
}

df = (spark \
.readStream \
.format("cloudFiles") \
.options(**cloudfile) \
.schema(schema) \
.load(f"wasbs://{container_name}@{storage_account_name}.blob.core.windows.net/")
)

(df.writeStream \
.format("delta") \
.option("checkpointLocation","/FileStore/tables/stream_checkpoint") \
.start("companywiseAnalysis")
)

I cannot understand the issue here. could you help me here

Azure Databricks
Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
1,843 questions
Azure Event Grid
Azure Event Grid
An Azure event routing service designed for high availability, consistent performance, and dynamic scale.
294 questions
{count} votes

1 answer

Sort by: Most helpful
  1. Ken Garber 1 Reputation point
    2022-08-11T18:06:34.54+00:00

    I was seeing a similar issue with our ADLS gen2 account where the data was landing. The solution for us was to enable the Allow Azure services on the trusted services list to access this storage account setting on the storage account. The account was pretty closed off with the Enabled from selected virtual networks and IP addresses setting enabled to specified VNets and Subnets.

    0 comments No comments