Author: Sai Kumar

How to Use Python For Loop ?

In this article, you’ll learn how to use Python for loop (Range Collection, String, Collections)? Using Python For Loop on range collection: Using Python For Loop in String: Using Python For Loop on Collections :   For any queries or doubts Ask Questions in 24Turorials Forum.

How to generate DDL(create statement) with columns using Python[code snippets]

Data loading is the initial step in Big Data Analytics world, you are supposed to push all the data to Hadoop first and then you can start working on analytics. When loading data to Hadoop environment, in some cases you will be getting data in the form of flat files. Once the data is loaded, if you want to view data or query this data we need to create HIVE table on top of that data. So it is obvi...

All about Python Classes – Demo with examples

Python Classes are all types – Class Definitions Class Initialization Class Methods

Deep dive into Partitioning in Spark – Hash Partitioning and Range Partitioning

Partitions- The data within an RDD is split into several partitions. Properties of partitions: – Partitions never span multiple machines, i.e., tuples in the same partition are guaranteed to be on the same machine. – Each machine in the cluster contains one or more partitions. – The number of partitions to use is configurable. By default, it equals the total number of cores on al...

Spark runtime Architecture – How Spark Jobs are executed

How Spark Jobs are Executed- A Spark application is a set of processes running on a cluster. All these processes are coordinated by the driver program. The driver is: -the process where the main() method of your program run. -the process running the code that creates a SparkContext, creates RDDs, and stages up or sends off transformations and actions. These processes that run computations and stor...

CUT command in Unix/Linux with examples

Cut Command: – CUT is used to process data in file. – Works only on file having column formatted data Command 1: Display particular position character cut -c3 file.txt Command 2: Range of characters cut -c3-8 file.txt cut -c3- file.txt cut -c-10 file.txt Command 3: Display Columns after seperation cut -d “|” -f2 file.txt cut -d “|” -f2-3 file.txt cut -d “|...

GREP command in Unix/Linux with examples

grep – Global Regular Expression Parser It is used to search data in one/more files. Command 1: Search pattern in file - grep hello file.txt - grep sai file.txt file2.txt Command 2: Search pattern in current folder with all txt extensions. grep 1000 *.txt Command 3: Search data in all files in current folder grep 1000 *.* Command 4: Search ignoring case[-i] grep "Sai" file.txt (case sensitiv...

SED command in Unix/Linux with examples

SED – Stream Editor Used to display & editing data Editing options are – Insertion/Updation/Deletion 2 Types of Operations ——————— – Lines Addressing – Context Addressing Line Addressing- Command 1: Display line multiple times sed '2p' file.txt sed -n '3p' file.txt (specific line => -n) sed -n '5p' file.txt Command 2: Display...

All about AWK command in Unix – Part 1

AWK – select column data -Search data in file and print data on console -Find data of specific columns -Format output data -Used on file with bulk of data for searching, conditional executions, updating, filtering Command 1 ——— Print specific columns awk '{print $1}' file.txt by default TAB seperator awk '{print $1 "--" $2}' file.txt Command 2 – ———&...

All about AWK command in Unix – Part 2

Command 11 – ———- Find text at the start of line [ ^ ] awk -F "|" '$2-/^s/{print $0}' tabfile.txt Command 12 – ———- Find text at the ent of line [ $ ] awk -F "|" '$2 -/n$/{print $0}' file1.txt Command 13 – ———- perform condition check using if awk -F "|" '{if ($3>2000) print $0;}' file2.txt Command 14 – ——...

Lost Password

Register

Do NOT follow this link or you will be banned from the site!