Shell

Scripting

Presented by:
Moataz Ahmed
Quick
Revision
Unix concept

• Directory Hierarchy

• Piping and Redirection

• Processes

3
Basic UNIX commands
• cd
• ls
• pwd
• mkdir
• rmdir
• find
• rm
• cat
• more
• head
• tail
• grep
• cut
• sort
• ….

4
Useful Commands
• split
• split a file into n line chunks
• diff
• compare two files
• line
• read the first line
• uniq
• select unique lines or rows

5
Useful Commands (cont.)

• join
• joining two files, matching row by row
• paste
• paste multiple files, column by column
• tr
• transform character by character

6
Examples
$ split -10 /etc/passwd
$ls
xaa xad
xab xae
xac xaf
$ line < /etc/passwd
root:x:0:1:Super-User:/:/sbin/sh
$ uniq names uniq-names
$ paste names uniq-names
sherine sherine
sherine root
root mdesouky
mdesouky
mdeouky

7
Streamlined
Editor sed
and
The Awk Utility
The Streamlined Editor

• What is sed?
• How sed works
• Addressing
• Commands
• Examples

9
What is sed?

• It is a streamline, non-interactive editor.

• It perform the same kind of tasks as in

vi.

• It doesn’t change your file unless the

output is saved with shell redirection.

10
How does sed Work?

• The sed editor process a file (input) one line at

a time and sends its output to the screen.

• The sed stores the line it process in a buffer

and once processing is finished the line is sent
to the screen (unless command was delete) and
the next line is read and the process is repeated
until the last line is reached.

11
Addressing
• Addressing is used to determine
which lines to be edited.
• The addressing format can be
• Number
• Regular expression

• Both

* Number represents a line number.

12
Commands
• The sed commands tell sed what to
do with the line:
• Print it
• Remove it
• Change it
• ….
• The sed format
sed ‘command’ filename

13
Examples
• To print lines contain the pattern root
$sed ‘/root/p’ myfile
sherine
maha
root
root
mdesouky
• To suppresses the default behavior of the
sed
$sed –n ‘/root/p’ myfile
root

14
Examples (cont.)

• To delete the third line

$sed ‘3d’ myfile
sherine
maha
mdesouky
• To delete from line 3 to the end
$sed ‘3,$d’ myfile
sherine
maha

15
 Examples (cont.)

• To delete the last line

$sed ‘$d’ myfile

sherine
maha
root

• To delete lines from 1 to 3

$sed ‘1,3d’ myfile

mdesouky

• To delete lines containing root pattern

$sed ‘/root/d’ myfile

sherine
maha
mdesouky

16
Examples (cont.)

• To substitute sherine by sbahader

$sed ‘s/sherine/sbahader/g’ myfile
sbahader
maha
root
mdesouky

$sed –n ‘s/sherine/sbahader/gp’ myfile

sbahader

• To print lines from maha to root

$sed –n ‘/maha/,/root/p’ myfile
maha
root

17
Examples (cont.)
• To print lines from 2 to the line that
begins with mdes
$sed –n ‘2,/^mdes/p’ myfile
maha
root
mdesouky

• To issue multi command

$sed –e ‘2d’ –e ‘s/sherine/sbahader/g’
myfile
sbahader
root
mdesouky

18
The awk Utility
• What is awk?
• What does awk stands for?
• The awk’s format
• Records and Fields
• Examples
• BEGIN Pattern
• END Pattern
• Conditional Expressions
• Loops
• Examples

19
What is Awk?
• awk is a programming language
used for manipulating data and
generating reports.

• awk scans a file line by line,

searching for lines that match a
specified pattern performing
selected actions

20
What does Awk stands for?
• awk stands for the first initials in the
last names of each authors of the
language, Alfred Aho, Peter
Weinberger, and Brian Kernighan

21
Awk’s Format
• The awk program consists of
• awk command
• Program instructions enclosed in quotes
• Input file or default stdin.

$awk ‘instructions’ inputfile

22
Records and fields
• By default, each line is called a record
and terminated with a new line.
• Record separators are by default
carriage return, stored in a built-in
variables ORS and RS.
• The $0 variable is the entire record.
• The NR variable is the record number.

23
Records and fields (cont.)

• Each record consists of words called

fields which by default separated
by white spaces.
• NF variables contains the number of
fields in a record
• FS variable holds the input field
separator, space/tab is the default.

24
Examples
• To print the first filed of the file as the default
delimiter is white space, you have to specify the
delimiter
$awk –F: ‘{print $1}’ /etc/passwd
root
daemon
sherine
…
$awk –F: ‘{print “Logname:”,$1}’ /etc/passwd
Logname:root
Logname:daemon
Logname:sherine
…

25
 Examples (cont.)

• To display the whole file (cat)

$awk ‘{print $0}’ /etc/passwd
root:x:0:1:Super-user:/:/sbin/sh
...

• To display the file numbered (cat -n)

$awk ‘{print NR,$0}’ /etc/passwd
1 root:x:0:1:Super-user:/:/sbin/sh
...

• To display number of fields (words) in each record (line)

$awk –F: ‘{print $0,NF}’ /etc/passwd
root:x:0:1:Super-user:/:/sbin/sh 7
...

26
BEGIN Pattern
• BEGIN Pattern is followed by an action block that
is executed before awk process any line from the
input file.

• BEGIN action is often used to change the value of

the built-in variables, FS, RS, and so forth to
assign initial values to user-defined variables and
print headers of titles.

• Example
$awk ‘BEGIN{FS=“:”; ORS=“\n\n”} {print $1,$2,$3}’
myfile

27
END Pattern
• END patterns are handled after all
lines of input have been processed.

• It does not match any input line

• Example:
• To print the number of lines in a file
$awk ‘END { print NR } ’ testfile

28
Conditional expressions
condition expression1 ? expression2 :expression3

if (expression1)
expression2
else
expression3

if (expression1){
statement; statement;...
}
else if (expression2){
statement; statement;...
}
else {
statement
}

29
Relational Operators
Operator Meaning
< Less than

<= Less than and equal

== Equal to
!= Not equal to
>= Greater than and equal
> Greater than
~ Match regular
expression
!~ Not matched by regular
expression

30
Loops
• while Loop
•Example
$awk –F: ’{i=1; while (i<NF) {print
NF,$i;i++}}’ /etc/passwd

• For Loop
•Example
$awk ‘{for ( i=1 ; i<NF; i++) print
NF,$i}’ /etc/passwd

31
Examples (cont.)

• The variable max value is set

according to compression
between the first 2 fields:
$ awk ‘{if ($1>$2)
max=$1;
else
max=$2;
print max}’ testing

• Arithmetical Operations
$ awk ‘{if ($1*$2>100) print $0}’ testing

32
Examples (cont.)

• To display line 4 and 5 only

$awk ‘{if (NR==4 || NR==5)
print NR”:”$0
}’ /etc/passwd

33