PySpark / Spark

PySpark – Components

PySpark Core Components includes –

  1. Spark Core – All functionalities built on top of Spark Core. Contains classes like SparkContext, RDD
  2. Spark SQL – Gives API for structured data processing. Contains important classes like SparkSession, DataFrame, DataSet.
  3. Spark Streaming – Gives functionality for Streaming data processing using micro-batching technique. Contains classes like Streaming Context, DStream
  4. Spark ML – Provides API to implement Machine learning algorithms.

Share This Post

Lost Password


24 Tutorials