Databricks runtime version 14.3 throwing error

Amit Singh 60

Hello,

I have multiple databricks job reporting failures when I use Databricks runtime version 14.3 (14.3.x-scala2.12)

The same job passes if I use Databricks runtime version 12.2
Attached is the error seen in stderr.log of the cluster, also mentioned here

ANTLR Tool version 4.8 used for code generation does not match the current runtime version 4.9.3

Exception in thread "maintenance-auto-compute-retry-thread-1724726451743" java.lang.InterruptedException

...

...
chown: invalid group: ‘:spark-users’

chmod: cannot access '/databricks/sparkconnect/grpc.sock': No such file or directory

Please share what could be the possible reason and fix?

Thanks

PRADEEPCHEEKATLA-MSFT 90,146 Reputation points Microsoft Employee

2024-08-27T10:14:30.4433333+00:00
@Amit Singh - Thanks for the question and using MS Q&A paltform.

It seems like the issue is related to the version of ANTLR Tool used for code generation. The error message indicates that the version used for code generation is 4.8, while the current runtime version is 4.9.3. This mismatch could be causing the issue.

To fix this issue, you can try upgrading the ANTLR Tool version to 4.9.3. You can do this by adding the following line to your build.sbt file:

libraryDependencies += "org.antlr" % "antlr4" % "4.9.3"

If this does not work, you can try downgrading the runtime version to 12.2, which you mentioned is working fine.

Regarding the errors related to chown and chmod, it seems like there might be some permission issues. You can try checking the permissions of the files and directories mentioned in the error message and make sure that the user running the job has the necessary permissions to access them.

Hope this helps. Do let us know if you any further queries.
PRADEEPCHEEKATLA-MSFT 90,146 Reputation points Microsoft Employee

2024-08-28T10:40:53.38+00:00

@Amit Singh - We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. In case if you have any resolution please do share that same with the community as it can be helpful to others. Otherwise, will respond with more details and we will try to help.

Your answer