Spark

Word count program in Spark

WordCount in Spark

WordCount program is like basic hello world program when it comes to Big data world. Below is program to achieve wordCount in Spark with very few lines of code.

[code lang=”scala”]val inputlines = sc.textfile("/users/guest/read.txt")
val words = inputlines.flatMap(line=>line.split(" "))
val wMap = words.map(word => (word,1))
val wOutput = wMap.reduceByKey(_ + _)
wOutput.saveAsTextFile("/users/guest/")[/code]

Share This Post

An Ambivert, music lover, enthusiast, artist, designer, coder, gamer, content writer. He is Professional Software Developer with hands-on experience in Spark, Kafka, Scala, Python, Hadoop, Hive, Sqoop, Pig, php, html,css. Know more about him at www.24tutorials.com/sai

Lost Password

Register

Do NOT follow this link or you will be banned from the site!