In some cases where you applied Joins in the spark application, you might want to know the time taken to complete the particular join. Below code snippet might come in handy to achieve so.
import java.util.Date val curent = new Date().getTime println(curent) Thread.sleep(30000) val end = new Date().getTime println(end) println("time taken "+(end-curent).toFloat/60000 + "mins")
Output:
import java.util.Date
curent: Long = 1520502573995
end: Long = 1520502603996
time taken 0.5000167mins
All you need to do is get current time before method starts and get current time after method ends, then calculate the difference to get total time taken to complete that particular method.
Hope this code snippet helps!!