Working With Hive Metastore in Apache Spark

In this video we will understand how to work with Hive Metastore in Apache Spark.

  • Spark Version – 2.4
  • Language – Scala

Objectives

  • How to read a hive table in Spark
  • Create a new table in Hive metastore using saveAsTable.
  • Create a partitioned table in hive using saveAsTable API
  • Create a new table using create table command
  • Write data in Overwrite mode
  • Write data in Append mode.

Downloading the practice dataset

In this lecture we are using RETAIL DB database. You can download the practice dataset from our Github repository.

YouTube player

Want to learn how we can work with different file formats ( parquet, JSON, Avro, ORC )using Spark SQL module? What out our playlist on youtube . Don’t forget to Subscribe 🙂

https://www.youtube.com/playlist?list=PLxPiYXz4lGTO5YZIX05uvgOOCkR_brJ_P