error: not found: value EventHubsConf

Question

Hi Everyone,
I was trying to do some real-time stream analytics using databricks and azure event hub but I keep running into this error even when I have configured everything well.
Here is the code snippet :

import java.util._
import scala.collection.JavaConverters._
import com.microsoft.azure.eventhubs._
import org.apache.spark.eventhubs.EventHubsConf._
import java.util.concurrent._
import org.apache.spark.sql.functions._
import org.apache.spark.sql.types._

//Read the data from the csv files
var tripdata = spark.read.option("header", true).csv(wasbsPath).limit(10000).rdd.collect.toList

//Set up Connection to Azure Event Hubs
val connStr = new ConnectionStringBuilder()
.setNamespaceName(namespaceName)
.setEventHubName(eventHubName)
.setSasKeyName(sasKeyName)
.setSasKey(sasKey)

val pool = Executors.newScheduledThreadPool(1)
val eventHubClient = EventHubClient.create(connStr.toString(), pool)

def sendEvent(message: String) = {
val messageData = EventData.create(message.getBytes("UTF-8"))
eventHubClient.get().send(messageData)
};

val customEventhubParameters = EventHubsConf(connStr.toString()).setMaxEventsPerTrigger(1) //set the maximun event at a time

// We will create a function to send records to Event Hubs. This is done to simulate a realtime stream of events. Whenever we use our incoming stream, we can call sendingEvents() function to send fresh events so that our analysis is performed on a realtime stream.

def sendingData(count : Int) : Unit ={
// Send stream to Event Hubs
for( a <- 0 to (tripdata.length - 1)){
sendEvent(tripdata(a).toString)
}
}

command-1988148540822794:29: error: not found: value EventHubsConf
val customEventhubParameters = EventHubsConf(connStr.toString()).setMaxEventsPerTrigger(1) //set the maximun event at a time

Answer 1

Hello @snehil singh ,
Thanks for the ask and using the Microsoft Q&A platform .

Are you not missing the consumer group here ?

EventHubsConf(connStr.toString())
.setConsumerGroup)"your consumer group")
.setMaxEventsPerTrigger(1)

Thanks
Himanshu
Please do consider to click on "Accept Answer" and "Up-vote" on the post that helps you, as it can be beneficial to other community members

Answer 2

Hi @HimanshuSinha-msft
I tried after creating new consumer group and added that to my command but it still shows the same error