Azure Event Hubs
An Azure real-time data ingestion service.
706 questions
The auto.register.schemas option does not work with JSON schema registry and kafka
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(281.6, 39%, 37%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EBK%3C/text%3E%3C/svg%3E)
Bill Kalantzis
0
Reputation points
In the code sample provided for the event hub schema registry for JSON and kafka producer, the auto.register.schemas property is not working.
I have created a new JSON schema group with no schemas, set the property auto.register.schemas=false, then run the sample application https://github.com/Azure/azure-schema-registry-for-kafka/blob/master/java/json/samples/producer/src/main/java/com/azure/schemaregistry/samples/producer/KafkaJsonSpecificRecord.java#L34.
I expect some type of error since my schema group contains no schemas when the sample program runs. Instead the schema gets created. This happens regardless if auto.register.schemas is false. I have verified this works for the avro implementation and am wondering if this is a bug in the JSON implementation
{count} votes
-
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(16, 19%, 11%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMS%3C/text%3E%3C/svg%3E)
Miguel Sa 0 Reputation points2023-08-18T21:12:54.97+00:00 The
auto.register.schemasoption is typically used with the Confluent Schema Registry in the context of Apache Kafka. This option allows you to automatically register schemas in the Schema Registry when a new schema is encountered during message production.The behavior you're describing, where the
auto.register.schemasoption doesn't work with the JSON schema registry and Kafka, might be due to configuration issues, version compatibility problems, or potentially even a bug in the software.Here are some steps you could consider taking to troubleshoot the issue:
Check Configuration: Double-check your Kafka producer configuration and ensure that the
auto.register.schemasoption is correctly set. Ensure that you have correctly configured the URL of the Schema Registry.Schema Compatibility: Make sure the schema format you're using is compatible with the Schema Registry you are using. The Confluent Schema Registry typically uses the Avro schema format, so if you're using a different schema format (like JSON Schema), there might be compatibility issues.
Version Compatibility: Ensure that the versions of your Kafka client library, Schema Registry, and any related components (like the JSON Schema registry, if that's what you're using) are compatible with each other. Incompatibilities between versions can lead to unexpected behavior.
Logging and Error Messages: Look at the logs of your Kafka producer and Schema Registry to see if there are any error messages or warnings that could provide more insight into the issue.
Community and Documentation: Check the official documentation for both the Kafka client library you're using and the Schema Registry you're using. You might find specific guidance on how to use the
auto.register.schemasoption or any known issues related to its usage.Community Forums and Issue Tracking: If you're still having trouble, consider reaching out to the community forums or issue trackers for the Kafka client library and the Schema Registry you're using. Other users or developers might have encountered similar issues and could provide guidance or solutions.
Remember that the information provided here is based on the state of things as of September 2021. For the most accurate and up-to-date guidance, I recommend checking the latest documentation, release notes, and community resources for the specific software components you're working with.
-
Bill Kalantzis 0 Reputation points
2023-08-25T20:58:59.0333333+00:00 I'm using the Event Hub Schema Registry and the code provided in the Azure Schema Registry documentation:
Again, the code used is referenced in the azure documentation:
https://learn.microsoft.com/en-us/azure/event-hubs/schema-registry-json-schema-kafka
-
PRADEEPCHEEKATLA 90,596 Reputation points2023-08-31T05:16:29.74+00:00 @Bill Kalantzis - If you have a bug or an idea, browse the open issues before opening a new one.
-
PRADEEPCHEEKATLA 90,596 Reputation points
2023-09-04T06:57:17.3733333+00:00 @Bill Kalantzis - We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. In case if you have any resolution please do share that same with the community as it can be helpful to others. Otherwise, will respond with more details and we will try to help.
Sign in to comment
Sign in to answer