ASP.NET Core
A set of technologies in the .NET Framework for building web applications and XML web services.
4,560 questions
This issue is closed Hangfire has fixed this issue. Thank you.
Hangfire is not working for CosmosDB after migrating from MongoDB giving error: A bulk write operation resulted in one or more errors. WriteErrors: [ { Category : "DuplicateKey", Code : 11000, Message : "E11000 duplicate key error collection: Hangfire
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(169.60000000000002, 20%, 26%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EMP%3C/text%3E%3C/svg%3E)
Murtuza Pithawala
0
Reputation points
We were using MongoDb in our project and Hangfire was working perfectly fine over there but after we have shifted to CosmosDb Hangfire has stopped working, we are using MongoClient only to connect to CosmosDB by passing CosmosDB connection string to MongoClient everything is fine with this MongoClient object but Hangfire is not working with this. It is creating Hangfire default collections into the CosmosDB and when we are trying to manually execute any of the jobs from Hangfire dashboard it gives us this error:
fail: Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware[1] An unhandled exception has occurred while executing the request. MongoDB.Driver.MongoBulkWriteException`1[MongoDB.Bson.BsonDocument]: A bulk write operation resulted in one or more errors. WriteErrors: [ { Category : "DuplicateKey", Code : 11000, Message : "E11000 duplicate key error collection: bomHangfire2.core.hangfire.jobGraph. Failed _id or unique index constraint." } ]. at MongoDB.Driver.MongoCollectionImpl`1.BulkWrite(IClientSessionHandle session, IEnumerable`1 requests, BulkWriteOptions options, CancellationToken cancellationToken) at MongoDB.Driver.MongoCollectionImpl`1.<>c__DisplayClass30_0.
{count} votes
-
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(211.20000000000002, 61%, 30%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EBS%3C/text%3E%3C/svg%3E)
Bruce (SqlWork.com) 65,316 Reputation points2023-09-06T16:45:13.48+00:00 cosmos db requires the _id key be unique. it can autogenerate a guid if not set, otherwise your code must supply a unique value.
hangfire has a cosmos db example:
https://github.com/Hangfire-Mongo/Hangfire.Mongo/tree/master/src
-
Murtuza Pithawala 0 Reputation points
2023-09-07T10:46:53.2233333+00:00 Tried hangfire sample of CosmosDb example, but after running that project it is giving the above error "A bulk write operation resulted in one or more errors"
-
Bruce (SqlWork.com) 65,316 Reputation points
2023-09-07T19:53:45.6233333+00:00 looks like you opened an issue on the GitHub site.
your only other option if you don't have support from hangfire, would be to clone the project, build the library and use it instead of the nuget package. then debug. once you have a fix, do a pull request.
I don't use hang fire, but a quick look at the code shows it uses jobid for _id. in mongo db this field must also be unique. so somehow in your implementation it generating duplicate jobids.
-
Oury Ba-MSFT 19,101 Reputation points Microsoft Employee2023-09-18T19:16:03.7766667+00:00 @Murtuza Pithawala Thank you for reaching out.
Sorry for the delay in my response.
Can you try our vCore architecture and let us know if you see the same issue?
Regards,
Oury
-
Oury Ba-MSFT 19,101 Reputation points Microsoft Employee
2023-09-19T20:12:11.07+00:00 @Murtuza Pithawala Checking if you had the chance to test it with our vcore architecture?
Regards,
Oury
-
Murtuza Pithawala 0 Reputation points
2023-09-20T05:52:27.81+00:00 @Oury Ba-MSFT Sorry, but didn't tried vCore architecture as it looks like it is related to database but right now we are not changing our database and if I am wrong you can correct me.
-
Oury Ba-MSFT 19,101 Reputation points Microsoft Employee
2023-09-22T17:18:31.4166667+00:00 I would suggest creating a support ticket so we can further investigate this.
Regards,
Oury
Sign in to comment
2 answers
Sort by: Most helpful
-
Murtuza Pithawala 0 Reputation points
2023-10-05T09:12:31.4733333+00:00 -
Oury Ba-MSFT 19,101 Reputation points Microsoft Employee
2023-10-06T15:39:01.99+00:00 Nice!
I'm glad also that hangfire was able to resolve this issue and it is not from our end. Thank you for posting your solution so that others experiencing the same thing can easily reference this! Since the Microsoft Q&A community has a policy that "The question author cannot accept their own answer. They can only accept answers by others", I'll repost your solution in case you'd like to "Accept" the answer.
Sign in to comment -
-
Oury Ba-MSFT 19,101 Reputation points Microsoft Employee
2023-10-06T15:39:02.79+00:00 @Murtuza Pithawala Thank you for reaching out.
Problem:
Hangfire is not working for CosmosDB after migrating from MongoDB giving error: A bulk write operation resulted in one or more errors. WriteErrors: [ { Category : "DuplicateKey", Code : 11000, Message : "E11000 duplicate key error collection: Hangfire
Solution:
Customer contacted Hangfire who fixed the issue.
Regards,
Oury