Note
Access to this page requires authorization. You can try signing in or changing directories.
Access to this page requires authorization. You can try changing directories.
A transform for any type that partitions by a hash of the input column.
Note
This function can be used only in combination with DataFrameWriterV2.partitionedBy method.
Syntax
from pyspark.sql.functions import partitioning
partitioning.bucket(numBuckets, col)
Parameters
| Parameter | Type | Description |
|---|---|---|
numBuckets |
pyspark.sql.Column or int |
The number of buckets. |
col |
pyspark.sql.Column or str |
Target date or timestamp column to work on. |
Examples
from pyspark.sql.functions import partitioning
df.writeTo("catalog.db.table").partitionedBy(
partitioning.bucket(42, "ts")
).createOrReplace()