File(filePath).exists does not work in Azure databricks

Question

Hi,

How to find if file exists in a path in the data lake?

Regards
Rajaniesh

Answer 1

@Rajaniesh, Welcome to the Microsoft Q&A platform.

From the error message, java.lang.Exception: Could not find file xxx /mnt/raw/File.json, looks like you are passing incorrect path.

Make sure the path exists and also try the steps mentioned below and do let us know the status.

Hope this helps. Do let us know if you any further queries.

----------------------------------------------------------------------------------------

Do click on "Accept Answer" and Upvote on the post that helps you, this can be beneficial to other community members.

Answer 2

Thanks Pradeep but this is the code used to read the file with spark.read which works fine for me as well. my path is also correct the but it does not works.

So I made this change

import java.io.File
import scala.io.Source
import scala.util.parsing.json.JSON

class SchemaJsonUtil(val schemaFilePath: String) extends java.io.Serializable {

private def loadFile(filePath: String): Map[String, List[String]] = {
var FileLines=""
val filename = ("/dbfs/").concat(schemaFilePath)
for (line <- Source.fromFile(filename).getLines()) {
FileLines=FileLines.concat(line)
}
val jsonString = FileLines
val jsonMap: Map[String, Any] = JSON.parseFull(jsonString).get.asInstanceOf[Map[String, Any]]
jsonMap.map { case(fix, typos) =>
(fix, typos.asInstanceOf[List[String]])
}.toMap
}

Still the issue remains same that how do we check if the path passed to the loadfile function is valid and file exists there or not? Since it is not working I removed the error checking code.Please let me know how do we find if the path is valid?

Regards
Rajaniesh