Have you contacted the third party software support team and asked them for help? That would seem to be first choice.
the application gets the license from a licensing service on the Windows Server.
The third party software uses a connection manager that connects directly to the SQL on the Server
Can you clarify the data flow so that we can understand where the authentication error might occur?
Where does this "connection manager" run? Is it a separate process that runs on the users pc along with the 3rd party app? Or is it integrated with the client app? Or does it run on the server?
Using SQL Management Studio, review the accounts that are allowed to access the database. Is each user defined? Or is there a single "application account"? Are you using SQL logins, or Windows users?
https://www.guru99.com/sql-server-create-user.html
Without Active Directory, I would expect that you would have to use SQL logins. And if didn't go through some process where you registered every user, then there would be a single SQL login that every instance of the "connection manager" would use to access the database.
Have you checked the SQL logs for login failures? Does the license service generate a log? Does the connection manager generate a log?