Manipulating String columns in Dataframe

In this video we will understand how to manipulate the String columns in Dataframe

  • Spark Version – 2.4
  • Language – Scala

Objectives

We will understand the basic string functions

  • concat_ws
  • Lower / lower
  • regexp_replace
  • split
  • substring

Downloading the practice dataset

In this lecture we are using RETAIL DB database. You can download the practice dataset from our Github repository.

YouTube player

Want to learn how we can work with different file formats ( parquet, JSON, Avro, ORC )using Spark SQL module? What out our playlist on youtube . Don’t forget to Subscribe 🙂

https://www.youtube.com/playlist?list=PLxPiYXz4lGTO5YZIX05uvgOOCkR_brJ_P