error: not found: value EventHubsConf

snehil singh 21 Reputation points
2021-02-08T10:56:02.347+00:00

Hi Everyone,
I was trying to do some real-time stream analytics using databricks and azure event hub but I keep running into this error even when I have configured everything well.
Here is the code snippet :

import java.util._
import scala.collection.JavaConverters._
import com.microsoft.azure.eventhubs._
import org.apache.spark.eventhubs.EventHubsConf._
import java.util.concurrent._
import org.apache.spark.sql.functions._
import org.apache.spark.sql.types._

//Read the data from the csv files
var tripdata = spark.read.option("header", true).csv(wasbsPath).limit(10000).rdd.collect.toList

//Set up Connection to Azure Event Hubs
val connStr = new ConnectionStringBuilder()
.setNamespaceName(namespaceName)
.setEventHubName(eventHubName)
.setSasKeyName(sasKeyName)
.setSasKey(sasKey)

val pool = Executors.newScheduledThreadPool(1)
val eventHubClient = EventHubClient.create(connStr.toString(), pool)

def sendEvent(message: String) = {
val messageData = EventData.create(message.getBytes("UTF-8"))
eventHubClient.get().send(messageData)
};

val customEventhubParameters = EventHubsConf(connStr.toString()).setMaxEventsPerTrigger(1) //set the maximun event at a time

// We will create a function to send records to Event Hubs. This is done to simulate a realtime stream of events. Whenever we use our incoming stream, we can call sendingEvents() function to send fresh events so that our analysis is performed on a realtime stream.

def sendingData(count : Int) : Unit ={
// Send stream to Event Hubs
for( a <- 0 to (tripdata.length - 1)){
sendEvent(tripdata(a).toString)
}
}

command-1988148540822794:29: error: not found: value EventHubsConf
val customEventhubParameters = EventHubsConf(connStr.toString()).setMaxEventsPerTrigger(1) //set the maximun event at a time

Azure Event Hubs
Azure Event Hubs
An Azure real-time data ingestion service.
570 questions
Azure Databricks
Azure Databricks
An Apache Spark-based analytics platform optimized for Azure.
1,972 questions
Azure Stream Analytics
Azure Stream Analytics
An Azure real-time analytics service designed for mission-critical workloads.
335 questions
0 comments No comments
{count} votes

Accepted answer
  1. HimanshuSinha-msft 19,381 Reputation points Microsoft Employee
    2021-02-08T22:25:44.867+00:00

    Hello @snehil singh ,
    Thanks for the ask and using the Microsoft Q&A platform .

    Are you not missing the consumer group here ?

    EventHubsConf(connStr.toString())
    .setConsumerGroup)"your consumer group")
    .setMaxEventsPerTrigger(1)

    Thanks
    Himanshu
    Please do consider to click on "Accept Answer" and "Up-vote" on the post that helps you, as it can be beneficial to other community members

    1 person found this answer helpful.
    0 comments No comments

1 additional answer

Sort by: Most helpful
  1. snehil singh 21 Reputation points
    2021-02-09T06:28:57.547+00:00

    Hi @HimanshuSinha-msft
    I tried after creating new consumer group and added that to my command but it still shows the same error

    0 comments No comments