PySpark Core Components includes –
- Spark Core – All functionalities built on top of Spark Core. Contains classes like SparkContext, RDD
- Spark SQL – Gives API for structured data processing. Contains important classes like SparkSession, DataFrame, DataSet.
- Spark Streaming – Gives functionality for Streaming data processing using micro-batching technique. Contains classes like Streaming Context, DStream
- Spark ML – Provides API to implement Machine learning algorithms.