Spark

Word count program in Spark

WordCount in Spark

WordCount program is like basic hello world program when it comes to Big data world. Below is program to achieve wordCount in Spark with very few lines of code.

val inputlines = sc.textfile("/users/guest/read.txt")
val words = inputlines.flatMap(line=>line.split(" "))
val wMap = words.map(word => (word,1))
val wOutput = wMap.reduceByKey(_ + _)
wOutput.saveAsTextFile("/users/guest/")

Share This Post

Avatar
An Ambivert, music lover, enthusiast, artist, designer, coder, gamer, content writer. He is Professional Software Developer with hands-on experience in Spark, Kafka, Scala, Python, Hadoop, Hive, Sqoop, Pig, php, html,css. Know more about him at www.saikumar.me

Lost Password

Register