In this video we will understand how to manipulate dates in Dataframe using Spark API
- Spark Version – 2.4
- Language – Scala
Objectives
- Convert Unix epoch to a string timestamp – from_unixtime()
- Convert Unix epoch to a date – to_date()
- Extract year, month and day from timestamp
- Extract Hour, Minute and Seconds from timestamp
- Add current date and current timestamp
Downloading the practice dataset
In this lecture we are using RETAIL DB database. You can download the practice dataset from our Github repository.
Don’t forget to subscribe our YouTube Channel
Want to learn how we can work with different file formats ( parquet, JSON, Avro, ORC )using Spark SQL module? What out our playlist on youtube . Don’t forget to Subscribe 🙂
https://www.youtube.com/playlist?list=PLxPiYXz4lGTO5YZIX05uvgOOCkR_brJ_P
