In this video we will understand DataFrame abstraction in Spark.
- Spark Version – 2.4
- Language – Scala
Objectives
- What is DataFrame Abstraction
- Spark SQL Architecture
- What is SparkSession
- Understanding DataFrameReader
- Understanding DataFrame Writer
Data Preparation
In this lecture we are using RETAIL DB database. You can download the practice dataset from our Github repository.
Don’t forget to subscribe our YouTube Channel
Hungary for more ?
Want to learn how we can work with different file formats ( parquet, JSON, Avro, ORC )using Spark SQL module? What out our playlist on youtube . Don’t forget to Subscribe 🙂
https://www.youtube.com/playlist?list=PLxPiYXz4lGTO5YZIX05uvgOOCkR_brJ_P