Note
Access to this page requires authorization. You can try signing in or changing directories.
Access to this page requires authorization. You can try changing directories.
Creates a string column for the file name of the current Spark task.
Syntax
from pyspark.sql import functions as sf
sf.input_file_name()
Returns
pyspark.sql.Column: file names.
Examples
Example 1: Get input file name
import os
from pyspark.sql import functions as sf
path = os.path.abspath(__file__)
df = spark.read.text(path)
df.select(sf.input_file_name()).first()
Row(input_file_name()='file:/...')