Sometimes, we want to convert pyspark string to date format with Python.
In this article, we’ll look at how to convert pyspark string to date format with Python.
How to convert pyspark string to date format with Python?
To convert pyspark string to date format with Python, we can use the to_timestamp method.
For instance, we write
from pyspark.sql.functions import to_timestamp
df = spark.createDataFrame([("1997-02-28 10:30:00",)], ["t"])
r = df.select(to_timestamp(df.t, "yyyy-MM-dd HH:mm:ss").alias("dt")).collect()
to create a data frame with spark.createDataFrame.
Then we call select to convert the datetime string in the data frame to a timestamp with to_timestamp.
Conclusion
To convert pyspark string to date format with Python, we can use the to_timestamp method.
