Text files

You can process files with the text format option to parse each line in any text-based file as a row in a DataFrame. This can be useful for a number of operations, including log parsing. It can also be useful if you need to ingest CSV or JSON data as raw strings.

For more information, see text files.

Options

Use the .option() and .options() methods of DataFrameReader and DataFrameWriter to configure text data sources. For a complete list of supported options, see DataFrameReader text options and DataFrameWriter text options.