timestamp_diff

Gets the difference between the timestamps in the specified units by truncating the fraction part.

Syntax

from pyspark.databricks.sql import functions as dbf

dbf.timestamp_diff(unit=<unit>, start=<start>, end=<end>)

Parameters

Parameter	Type	Description
unit	literal string	This indicates the units of the difference between the given timestamps. Supported options are (case insensitive): "YEAR", "QUARTER", "MONTH", "WEEK", "DAY", "HOUR", "MINUTE", "SECOND", "MILLISECOND" and "MICROSECOND".
start	pyspark.sql.Column or str	A timestamp which the expression subtracts from endTimestamp.
end	pyspark.sql.Column or str	A timestamp from which the expression subtracts startTimestamp.

Returns

pyspark.sql.Column: the difference between the timestamps.

Examples

import datetime
from pyspark.databricks.sql import functions as dbf
df = spark.createDataFrame(
[(datetime.datetime(2016, 3, 11, 9, 0, 7), datetime.datetime(2024, 4, 2, 9, 0, 7))],
['ts1', 'ts2'])
df.select('*', dbf.timestamp_diff('year', 'ts1', 'ts2')).show()
df.select('*', dbf.timestamp_diff('WEEK', 'ts1', 'ts2')).show()
df.select('*', dbf.timestamp_diff('day', df.ts2, df.ts1)).show()