從給定陣列中移除所有等於元素的元素。
語法
from pyspark.sql import functions as sf
sf.array_remove(col, element)
參數
| 參數 | 類型 | Description |
|---|---|---|
col |
pyspark.sql.Column 或 str |
包含陣列的欄位名稱 |
element |
任意 | 元素或欄位表達式將從陣列中移除 |
退貨
pyspark.sql.Column： 一個新的欄位，該陣列從輸入欄位中排除給定值。
範例
範例 1：從簡單陣列中移除特定值
from pyspark.sql import functions as sf
df = spark.createDataFrame([([1, 2, 3, 1, 1],)], ['data'])
df.select(sf.array_remove(df.data, 1)).show()
+---------------------+
|array_remove(data, 1)|
+---------------------+
| [2, 3]|
+---------------------+
範例 2：從多個陣列中移除特定值
from pyspark.sql import functions as sf
df = spark.createDataFrame([([1, 2, 3, 1, 1],), ([4, 5, 5, 4],)], ['data'])
df.select(sf.array_remove(df.data, 5)).show()
+---------------------+
|array_remove(data, 5)|
+---------------------+
| [1, 2, 3, 1, 1]|
| [4, 4]|
+---------------------+
範例 3：移除陣列中不存在的值
from pyspark.sql import functions as sf
df = spark.createDataFrame([([1, 2, 3],)], ['data'])
df.select(sf.array_remove(df.data, 4)).show()
+---------------------+
|array_remove(data, 4)|
+---------------------+
| [1, 2, 3]|
+---------------------+
範例 4：從一個所有值都相同的陣列中移除一個值
from pyspark.sql import functions as sf
df = spark.createDataFrame([([1, 1, 1],)], ['data'])
df.select(sf.array_remove(df.data, 1)).show()
+---------------------+
|array_remove(data, 1)|
+---------------------+
| []|
+---------------------+
範例 5：從空陣列中移除一個值
from pyspark.sql import functions as sf
from pyspark.sql.types import ArrayType, IntegerType, StructType, StructField
schema = StructType([
StructField("data", ArrayType(IntegerType()), True)
])
df = spark.createDataFrame([([],)], schema)
df.select(sf.array_remove(df.data, 1)).show()
+---------------------+
|array_remove(data, 1)|
+---------------------+
| []|
+---------------------+
範例 6：從簡單陣列中移除欄位的值
from pyspark.sql import functions as sf
df = spark.createDataFrame([([1, 2, 3, 1, 1], 1)], ['data', 'col'])
df.select(sf.array_remove(df.data, df.col)).show()
+-----------------------+
|array_remove(data, col)|
+-----------------------+
| [2, 3]|
+-----------------------+