pyspark.sql.functions.levenshtein#

pyspark.sql.functions.levenshtein(left, right, threshold=None)[source]#

Computes the Levenshtein distance of the two given strings.

Added in version 1.5.0.

Changed in version 3.4.0: Supports Spark Connect.

Parameters:

leftColumn or column name: first column value. A column that evaluates to a string.
rightColumn or column name: second column value. A column that evaluates to a string.
thresholdint, optional: if set when the levenshtein distance of the two given strings less than or equal to a given threshold then return result distance, or -1. A column that evaluates to an integer.

Added in version 3.5.0.

Returns:

Column: Levenshtein distance as integer value. Returns a column that evaluates to an integer.

Examples

>>> from pyspark.sql import functions as sf
>>> df = spark.createDataFrame([('kitten', 'sitting',)], ['l', 'r'])
>>> df.select('*', sf.levenshtein('l', 'r')).show()
+------+-------+-----------------+
|     l|      r|levenshtein(l, r)|
+------+-------+-----------------+
|kitten|sitting|                3|
+------+-------+-----------------+

>>> df.select('*', sf.levenshtein(df.l, df.r, 2)).show()
+------+-------+--------------------+
|     l|      r|levenshtein(l, r, 2)|
+------+-------+--------------------+
|kitten|sitting|                  -1|
+------+-------+--------------------+