Subject Name: Unix and shell programming

Subject code: 15CS35

Author: P Ramesh Naidu

UNIX AND SHELL PROGRAMMING

[As per Choice Based Credit System (CBCS) scheme]
(Effective from the academic year 2015 -2016)
SEMESTER – III
Subject Code 15CS35 IA Marks 20

Number of Lecture Hours/Week 04 Exam Marks 80

Total Number of Lecture Hours 50 Exam Hours 03

CREDITS – 04

Course objectives: This course will enable students to

 Understand the UNIX Architecture, File systems and use of basic Commands.
 Use of editors and Networking commands.
 Understand Shell Programming and to write shell scripts.
 Understand and analyze UNIX System calls, Process Creation, Control & Relationship.
Module -1 Teaching
Hours

Introduction, Brief history. Unix Components/Architecture. Features of Unix. The UNIX Environment
and UNIX Structure, Posix and Single Unix specification. The login prompt. General features of Unix
commands/ command structure. Command arguments and options. Understanding of some basic 10Hours
commands such as echo, printf, ls, who, date, passwd, cal, Combining commands. Meaning of Internal
and external commands. The type command: knowing the type of a command and locating it. The man
command knowing more about Unix commands and using Unix online manual pages. The man with
keyword option and whatis. The more command and using it with other commands. Knowing the user
terminal, displaying its characteristics and setting characteristics. Managing the non-uniform behaviour
of terminals and keyboards. The root login. Becoming the super user: su command. The /etc/passwd and
/etc/shadow files. Commands to add, modify and delete users.

Topics from chapter 2 , 3 and 15 of text book 1,chapter 1 from text book
Module -2

Unix files. Naming files. Basic file types/categories. Organization of files. Hidden files. Standard
directories. Parent child relationship. The home directory and the HOME variable. Reaching required
files- the PATH variable, manipulating the PATH, Relative and absolute pathnames. Directory 10Hours
.
commands – pwd, cd, mkdir, rmdir commands. The dot (.) and double dots (..) notations to represent
present and parent directories and their usage in relative path names. File related commands – cat, mv,
rm, cp, wc and od commands. File attributes and permissions and knowing them. The ls command with
options. Changing file permissions: the relative and absolute permissions changing methods.
Recursively changing file permissions. Directory permissions.

Topics from chapters 4, 5 and 6 of text book 1

Module – 3

The vi editor. Basics. The .exrc file. Different ways of invoking and quitting vi. Different modes of vi.
Input mode commands. Command mode commands. The ex mode commands. Illustrative examples
Navigation commands. Repeat command. Pattern searching. The search and replace command. The set, 10Hours
map and abbr commands. Simple examples using these commands.

The shells interpretive cycle. Wild cards and file name generation. Removing the special meanings of
wild cards. Three standard files and redirection. Connecting commands: Pipe. Splitting the output: tee.
Command substitution. Basic and Extended regular expressions. The grep, egrep. Typical examples
involving different regular expressions.
Topics from chapters 7, 8 and 13 of text book 1. Topics from chapter 2 and 9 ,10 of text book 2
Module-4

Shell programming. Ordinary and environment variables. The .profile. Read and readonly commands.
Command line arguments. exit and exit status of a command. Logical operators for conditional
execution. The test command and its shortcut. The if, while, for and case control statements. The set and 10Hours
shift commands and handling positional parameters. The here ( << ) document and trap command.
Simple shell program examples. File inodes and the inode structure. File links – hard and soft links.
Filters. Head and tail commands. Cut and paste commands. The sort command and its usage with
different options. The umask and default file permissions. Two special files /dev/null and /dev/tty.

Topics from chapter 11, 12, 14 of text book 1,chapter 17 from text book2

.
Module-5

Meaning of a process. Mechanism of process creation. Parent and child process. The ps command with
its options. Executing a command at a specified point of time: at command. Executing a command
periodically: cron command and the crontab file.. Signals. The nice and nohup commands. Background 10Hours
processes. The bg and fg command. The kill command. The find command with illustrative example.
Structure of a perl script. Running a perl script. Variables and operators. String handling functions.
Default variables - $_ and $. – representing the current line and current line number. The range operator.
Chop() and chomp() functions. Lists and arrays. The @- variable. The splice operator, push(), pop(),
split() and join(). File handles and handling file – using open(), close() and die () functions.. Associative
arrays – keys and value functions. Overview of decision making loop control structures – the foreach.
Regular expressions – simple and multiple search patterns. The match and substitute operators.
Defining and using subroutines.
Topics from chapter 9 and 19 of text book 1. Topics from chapter 11 of reference book 1
Course outcomes:

After studying this course, students will be able to:

 Explain multi user OS UNIX and its basic features

 Interpret UNIX Commands, Shell basics, and shell environments
 Design and develop shell programming, communication, System calls and terminology.
 Design and develop UNIX File I/O and UNIX Processes.
 Perl script writing
Graduate Attributes (as per NBA)

1. Engineering Knowledge
2. Environment and Sustainability
3. Design/Development of Solutions
Question paper pattern:

The question paper will have ten questions.

There will be 2 questions from each module.
Each question will have questions covering all the topics under a module.
The students will have to answer 5 full questions, selecting one full question from each module.

Text Books:

1. Sumitabha Das., Unix Concepts and Applications., 4th Edition., Tata McGraw Hill

2. Behrouz A. Forouzan, Richard F. Gilberg : UNIX and Shell Programming- Cengage Learning – India
Edition. 2009.
Reference Books:

1. M.G. Venkatesh Murthy: UNIX & Shell Programming, Pearson Education.

2. Richard Blum , Christine Bresnahan : Linux Command Line and Shell Scripting Bible, 2ndEdition ,
Wiley,2014.
Module -1

Chapter 1. The UNIX Operating System

Introduction
This chapter introduces you to the UNIX operating system. We first look at what is an
operating system and then proceed to discuss the different features of UNIX that have
made it a popular operating system.

Objectives
 What is an operating system (OS)?
 Features of UNIX OS
 A Brief History of UNIX OS, POSIX and Single Unix Specification (SUS)

1. What is an operating system (OS)?

An operating system (OS) is a resource manager. It takes the form of a set of software
routines that allow users and application programs to access system resources (e.g. the
CPU, memory, disks, modems, printers, network cards etc.) in a safe, efficient and
abstract way.

For example, an OS ensures safe access to a printer by allowing only one application
program to send data directly to the printer at any one time. An OS encourages efficient
use of the CPU by suspending programs that are waiting for I/O operations to complete to
make way for programs that can use the CPU more productively. An OS also provides
convenient abstractions (such as files rather than disk locations) which isolate
application programmers and users from the details of the underlying hardware.

User Applications

Application System Utilities

Programs

System Call Interface

Operating System Kernel

Processor/Hardware

UNIX Operating system allows complex tasks to be performed with a few keystrokes. It
doesn’t tell or warn the user about the consequences of the command.
 Kernighan and Pike (The UNIX Programming Environment) lamented long ago that ―as
the UNIX system has spread, the fraction of its users who are skilled in its application has
decreased.‖ However, the capabilities of UNIX are limited only by your imagination.

2. A Brief History of UNIX

In the late 1960s, researchers from General Electric, MIT and Bell Labs launched a joint
project to develop an ambitious multi-user, multi-tasking OS for mainframe computers
known as MULTICS (Multiplexed Information and Computing System). MULTICS
failed, but it did inspire Ken Thompson, who was a researcher at Bell Labs, to have a go
at writing a simpler operating system himself. He wrote a simpler version of MULTICS
on a PDP7 in assembler and called his attempt UNICS (Uniplexed Information and
Computing System). Because memory and CPU power were at a premium in those days,
UNICS (eventually shortened to UNIX) used short commands to minimize the space
needed to store them and the time needed to decode them - hence the tradition of short
UNIX commands we use today, e.g. ls, cp, rm, mv etc.

Ken Thompson then teamed up with Dennis Ritchie, the author of the first C compiler in
1973. They rewrote the UNIX kernel in C - this was a big step forwards in terms of the
system's portability - and released the Fifth Edition of UNIX to universities in 1974. The
Seventh Edition, released in 1978, marked a split in UNIX development into two main
branches: SYSV (System 5) and BSD (Berkeley Software Distribution). BSD arose from
the University of California at Berkeley where Ken Thompson spent a sabbatical year. Its
development was continued by students at Berkeley and other research institutions.
SYSV was developed by AT&T and other commercial companies. UNIX flavors based
on SYSV have traditionally been more conservative, but better supported than BSD-
based flavors.

Until recently, UNIX standards were nearly as numerous as its variants. In early
days, AT&T published a document called System V Interface Definition (SVID).
X/OPEN (now The Open Group), a consortium of vendors and users, had one too, in
the X/Open Portability Guide (XPG). In the US, yet another set of standards, named
Portable Operating System Interface for Computer Environments (POSIX), were
developed at the behest of the Institution of Electrical and Electronics Engineers
(IEEE).
In 1998, X/OPEN and IEEE undertook an ambitious program of unifying the two
standards. In 2001, this joint initiative resulted in a single specification called the
Single UNIX Specification, Version 3 (SUSV3), that is also known as IEEE
1003.1:2001 (POSIX.1). In 2002, the International Organization for Standardization
(ISO) approved SUSV3 and IEEE 1003.1:2001.
Some of the commercial UNIX based on system V are:

 IBM's AIX
 Hewlett-Packard's HPUX
 SCO's Open Server Release 5
 Silicon Graphics' IRIS
 DEC's Digital UNIX
 Sun Microsystems' Solaris 2

Some of the commercial UNIX based on BSD are:

 SunOS 4.1.X (now Solaris)

 DEC's Ultris
 BSD/OS, 4.4BSD

Some Free UNIX are:

 Linux, written by Linus Torvalds at University of Helsinki in Finland.

 FreeBSD and NetBSD, a derivative of 4.4BSD

3. Features of UNIX OS
Several features of UNIX have made it popular. Some of them are:

Portable
UNIX can be installed on many hardware platforms. Its widespread use can be traced to
the decision to develop it using the C language.

Multiuser
The UNIX design allows multiple users to concurrently share hardware and software

Multitasking
UNIX allows a user to run more than one program at a time. In fact more than one
program can be running in the background while a user is working foreground.

Networking
While UNIX was developed to be an interactive, multiuser, multitasking system,
networking is also incorporated into the heart of the operating system. Access to another
system uses a standard communications protocol known as Transmission Control
Protocol/Internet Protocol (TCP/IP).

Organized File System

UNIX has a very organized file and directory system that allows users to organize and
maintain files.

Device Independence
UNIX treats input/output devices like ordinary files. The source or destination for file
input and output is easily controlled through a UNIX design feature called redirection.

Utilities
UNIX provides a rich library of utilities that can be use to increase user productivity.
 Unix Architecture

Users

Shell

Kernel

Hardware

System Calls

UNIX architecture comprises of two major components viz., the shell and the kernel. The
kernel interacts with the machine’s hardware and the shell with the user.

The kernel is the core of the operating system. It is a collection of routines written in C. It
is loaded into memory when the system is booted and communicates directly with the
hardware. User programs that need to access the hardware use the services of the kernel
via use of system calls and the kernel performs the job on behalf of the user. Kernel is
also responsible for managing system’s memory, schedules processes, decides their
priorities.

The shell performs the role of command interpreter. Even though there’s only one kernel
running on the system, there could be several shells in action, one for each user who’s
logged in. The shell is responsible for interpreting the meaning of metacharacters if any,
found on the command line before dispatching the command to the kernel for execution.

The File and Proces

A file is an array of bytes that stores information. It is also related to another file in the
sense that both belong to a single hierarchical directory structure.

A process is the second abstraction UNIX provides. It can be treated as a time image of
an executable file. Like files, processes also belong to a hierarchical structure. We will be
discussing the processes in detain in a subsequent chapter.

2. Locating Files
All UNIX commands are single words like ls, cd, cat, etc. These names are in lowercase.
These commands are essentially files containing programs, mainly written in C. Files are
stored in directories, and so are the binaries associated with these commands. You can
find the location of an executable program using type command:
$ type ls
 ls is /bin/ls
This means that when you execute ls command, the shell locates this file in /bin directory
and makes arrangements to execute it.

The Path
The sequence of directories that the shell searches to look for a command is specified in
its own PATH variable. These directories are colon separated. When you issue a
command, the shell searches this list in the sequence specified to locate and execute it.

3. Internal and External Commands

Some commands are implemented as part of the shell itself rather than separate
executable files. Such commands that are built-in are called internal commands. If a
command exists both as an internal command of the shell as well as an external one (in
/bin or /usr/bin), the shell will accord top priority to its own internal command with the
same name. Some built-in commands are echo, pwd, etc.

4. Command Structure
UNIX commands take the following general form:
verb [options] [arguments]
where verb is the command name that can take a set of optional options and one or more
optional arguments.
Commands, options and arguments have to be separated by spaces or tabs to enable the
shell to interpret them as words. A contiguous string of spaces and tabs together is called
a whitespace. The shell compresses multiple occurrences of whitespace into a single
whitespace.

Options
An option is preceded by a minus sign (-) to distinguish it from filenames.
Example: $ ls –l
There must not be any whitespaces between – and l. Options are also arguments, but
given a special name because they are predetermined. Options can be normally compined
with only one – sign. i.e., instead of using
$ ls –l –a –t
we can as well use,
$ ls –lat
Because UNIX was developed by people who had their own ideas as to what options
should look like, there will be variations in the options. Some commands use + as an
option prefix instead of -.

Filename Arguments
Many UNIX commands use a filename as argument so that the command can take input
from the file. If a command uses a filename as argument, it will usually be the last
argument, after all options.
Example: cp file1 file2 file3 dest_dir
rm file1 file2 file3
The command with its options and argumens is known as the command line, which is
considered as complete after [Enter] key is pressed, so that the entire line is fed to the
shell as its input for interpretation and execution.
Exceptions
Some commands in UNIX like pwd do not take any options and arguments. Some
commands like who may or may not be specified with arguments. The ls command can
run without arguments (ls), with only options (ls –l), with only filenames (ls f1 f2), or
using a combination of both (ls –l f1 f2). Some commands compulsorily take options
(cut). Some commands like grep, sed can take an expression as an argument, or a set of
instructions as argument.

5. Flexibility of Command Usage

UNIX provides flexibility in using the commands. The following discussion looks at how
permissive the shell can be to the command usage.

Combining Commands
Instead of executing commands on separate lines, where each command is processed and
executed before the next could be entered, UNIX allows you to specify more than one
command in the single command line. Each command has to be separated from the other
by a ; (semicolon).
wc sample.txt ; ls –l sample.txt
You can even group several commands together so that their combined output is
redirected to a file.
(wc sample.txt ; ls –l sample.txt) > newfile
When a command line contains a semicolon, the shell understands that the command on
each side of it needs to be processed separately. Here ; is known as a metacharacter.

Note: When a command overflows into the next line or needs to be split into multiple
lines, just press enter, so that the secondary prompt (normally >) is displayed and you can
enter the remaining part of the command on the next line.

Entering a Command before previous command has finished

You need not have to wait for the previous command to finish before you can enter the
next command. Subsequent commands entered at the keyboard are stored in a buffer (a
temporary storage in memory) that is maintained by the kernel for all keyboard input.
The next command will be passed on to the shell for interpretation after the previous
command has completed its execution.

6. man: Browsing The Manual Pages Online

UNIX commands are rather cryptic. When you don’t remember what options are
supported by a command or what its syntax is, you can always view man (short for
manual) pages to get online help. The man command displays online documentation of a
specified command.

A pager is a program that displays one screenful information and pauses for the user to
view the contents. The user can make use of internal commands of the pager to scroll up
and scroll down the information. The two popular pagers are more and less. more is the
Berkeley’s pager, which is a superior alternative to original pg command. less is the
standard pager used on Linux systems. less if modeled after a popular editor called vi and
is more powerful than more as it provides vi-like navigational and search facilities. We
can use pagers with commands like ls | more. The man command is configured to work
with a pager.
 7. Understanding The man Documentation
The man documentation is organized in eight (08) sections. Later enhancements have
added subsections like 1C, 1M, 3N etc.) References to other sections are reflected as SEE
ALSO section of a man page.

When you use man command, it starts searching the manuals starting from section 1. If it
locates a keyword in one section, it won’t continue the search, even if the keyword occurs
In another section. However, we can provide the section number additionally as argument
for man command.

For example, passwd appears in section 1 and section 4. If we want to get documentation
of passwd in section 4, we use,
$ man 4 passwd OR $ man –s4 passwd (on Solaris)

Understanding a man Page

A typical man page for wc command is shown below:

User Commands wc(1) NAME

wc – displays a count of lines, words and characters
in a file
SYNOPSIS
wc [-c | -m | -C] [-lw] [file ...] DESCRIPTION
The wc utility reads one or more input files and, by
default, writes the number of newline characters, words and bytes contained in
each input file to the standard output. The utility also writes a total count for all
named files, if more than one input file is specified.
OPTIONS
The following options are supported:
-c Count bytes.
-m Count characters.
-C same as –m.
-l Count lines.
-w Count words delimited by white spaces or new line characters ...
OPERANDS
The following operand is supported:
file A path name of an input file. If no file operands are specified, the
standard input will be used.
EXIT STATUS
See largefile(5) for the description of the behavior of wc when
encountering files greater than or equal
to 2 Gbyte (2 **31 bytes)
SEE ALSO
cksum(1), isspace(3C), iswalpha(3C), iswspace(3C), largefile(5), ...

A man page is divided into a number of compulsory and optional sections. Every
command doesn’t need all sections, but the first three (NAME, SYNOPSIS and
DESCRIPTION) are generally seen in all man pages. NAME presents a one-line
introduction of the command. SYNOPSIS shows the syntax used by the command and
DESCRIPTION provides a detailed description.

The SYNOPSIS follows certain conventions and rules:

 If a command argument is enclosed in rectangular brackets, then it is optional;
otherwise, the argument is required.
 The ellipsis (a set if three dots) implies that there can be more instances of the
preceding word.
 The | means that only one of the options shows on either side of the pipe can be
used.
All the options used by the command are listed in OPTIONS section. There is a separate
section named EXIT STATUS which lists possible error conditions and their numeric
representation.

Note: You can use man command to view its own documentation ($ man man). You can
also set the pager to use with man ($ PAGER=less ; export PAGER). To understand
which pager is being used by man, use $ echo $PAGER.

The following table shows the organization of man documentation.

Section Subject (SVR4) Subject (Linux)

1 User programs User programs
2 Kernel’s system calls Kernel’s system calls
3 Library functions Library functions
4 Administrative file formats Special files (in /dev)
5 Miscellaneous Administrative file formats
6 Games Games
7 Special files (in /dev) Macro packages and conventions
8 Administration commands Administration commands

8. Further Help with man –k, apropos and whatis

man –k: Searches a summary database and prints one-line description of the command.
Example:
$ man –k awk
awk awk(1) -pattern scanning and processing language nawk nawk(1) -pattern
scanning and processing language

apropos: lists the commands and files associated with a keyword.

Example:
$ apropos FTP
ftp ftp(1) -file transfer program
ftpd in.ftpd(1m) -file transfer protocol server
ftpusers ftpusers(4) -file listing users to be disallowed
ftp login privileges
whatis: lists one-liners for a command.
Example:
$ whatis cp
cp cp(1) -copy files

9. When Things Go Wrong

Terminals and keyboards have no uniform behavioral pattern. Terminal settings directly
impact the keyboard operation. If you observe a different behavior from that expected, when you press
certain keystrokes, it means that the terminal settings are different. In such cases, you should know
which keys to press to get the required behavior. The following table lists keyboard commands to try when
things go wrong.

Keystroke Function
or
command
[Ctrl-h] Erases text
[Ctrl-c] or Interrupts a command
Delete
[Ctrl-d] Terminates login session or a program that expects its input from
keyboard
[Ctrl-s] Stops scrolling of screen output and locks keyboard
[Ctrl-q] Resumes scrolling of screen output and unlocks keyboard
[Ctrl-u] Kills command line without executing it
[Ctrl-\] Kills running program but creates a core file containing the memory
image of the program
[Ctrl-z] Suspends process and returns shell prompt; use fg to resume job
[Ctrl-j] Alternative to [Enter]
[Ctrl-m] Alternative to [Enter]
stty sane Restores terminal to normal status

10.POSIX AND THE SINGLE UNIX SPECIFICATION

Dennis Ritchie's decision to rewrite UNIX in C didn't quite make UNIX very portable. UNIX
fragmentation and the absence of a single conforming standard adversely affected the development
of portable applications. First, AT&T created the System V Interface Definition (SVID). Later, XI
Open (now The Open Group), a consortium of vendors and users, created the X/Open Portability
Guide (XPG). Products conforming to this specification were branded UNIX95 , UNIX98 or
UNlX03 depending on the version of the specification.
Yet another group of standards, the Portable Operating System Interface for Computer Environments
(POSIX), were developed at the behest of the Institution of Electrical and Electronics Engineers
(IEEE). POSIX refers to operating systems in general, but was based on UNIX. Two of the
most- cited standards from the PO SIX family are known as POSIX.] and POSIX.2. POSIX.I
specifics the C application program interface-the system calls. POSIX.2 deals with the shell
and utilities.
In 2001, a joint initiative of X/Open and IEEE resulted in the unification of the two
standards. This is the Sillgle UNIX Specification, Version 3 (SUSV3). The "write once, adopt
everywhere" approach to this development means that once software has been developed on
any POSIX-
When you issue a command, the shell searches this list in the sequence specified to locate and execute it.
Note that this list also includes the current directory indicated by a singular dot at the end. The
following message shows that the netscape command is not available in any of these directories:
 $ netscape
ksh: netscape: not found
The Korn shell is running here and prints the message after failing to locate the file. This doesn't in
any way confirm that netscape doesn't exist on this system; it could reside in a different directory. In
that case we can still run it
• by changing the value of PATH to include that directory .
• by using a pathname (like /usr/local/bin/netscape if the command is located in/usr/l oca
1fbi n).

11. INTERNAL AND EXTERNAL COMMANDS

Since ls is a program or file having an independent existence in the /bin directory (or /usr/bin), it
is branded as an external command. Most commands are external in nature, but there are some
which arc not really found anywhere; and some which are normally not executed even if they are
in one of the directories specified by PATH. Take for instance the echo command:
$ type eeho
echo is a shell builtin
echo isn't an external command in the sense that, when you type echo, the shell won't look in its
PATH to locate it (even if it is there in fbi n). Rather, it will execute it from its own set of built-in
commands that arc not stored as separate tiles. These built-in commands, of which echo is a member, are
known as internal commands.
This program starts running for you when you log in, and dies when you log out. The shell is an
external command with a difference; it possesses its own set of internal commands. So if a command
exists both as an internal command of the shell as well as an external one (in /bin or /usr/bin), the
shell will accord top priority to its own internal command of the same name.
What you need to keep in mind is something that beginners often forget-the necessity of
providing spaces between the command and argument. If you have used DIR/P instead of DIR
IP in DOS, don't expect UNIX to be equally accommodating:
S 1s-1
bash: Is-1: command not found
Options can normally be combined with only one - sign, i.e., instead of
using

Is -1 -a -t
Is -lat Same as 1s -I -a - t

to obtain the same output. This facility reduces your typing load, which becomes significant
when you use commands with several options. The command parses (breaks up) the option
combination into separate options.
Because UNIX was developed by people who had their own ideas as to what options should look like,
there will invariably be exceptions [0 whatever rules we try to formulate. Some commands won't let
you combine options in the way you did just now. There are some that use + as an option prefix instead
of -.-. Some even use the =1
12. Basic Commands
Cal :
You can invoke the cal command to see the calendar of any specific month or a complete year. The
facility is totally accurate and takes into account the leap year adjustments that took place in the
year 1752. Let's have a look at its syntax drawn from the Solaris man page:

cal [ [month] year ]

Everything within r e c t a n g u l a r brackets is optional, So, cal can be used without arguments, in
which case it displays the calendar of the current month:
$ cal
April 2016
Su Mo Tu We Th Fr Sa
1 2
3 4 5 6 7 8 9
10 11 12 13 14 15 16
17 18 19 20 21 22 23
24 25 26 27 28 29 30

$ cal 2 2015
February 2015
Su Mo Tu We Th Fr Sa
1 2 3 4 5 6 7
8 9 10 11 12 13 14
15 16 17 18 19 20 21
22 23 24 25 26 27 28

Date :
To display or change the date.

'date' with no arguments prints the current time and date, in the format of the %c directive (described
below). If given an argument that starts with a +, date prints the current time and date (or the time and date
specified by the --date option, see below) in the format defined by that argument, which is the same as in the
strftime function.Except for directives, which start with %, characters in the format string are printed
unchanged.

>date

Mon Jan 23 01:37:51 PST 2012

date '+%m-%d-%Y'

01-23-2012
You can format the output of date command by using the %. Here %m for month, %d for day and %Y for
year.

bc: basic calculator

Arithmetic operations are the most common in any kind of programming language. Unix or linux operating
system provides the bc command .UNIX provides two types of calculators--a graphical object that
looks like one, and the text-based bc command
$ bc
12 + 5

17
[Ctrl-d]

echo:
This command is used in shell scripts to display diagnostic messages on the terminal or to issue prompts for
taking user input. we have used it in two ways:

1. To display a message

2. To evaluate shell variables

Echo was an external command, but all shells have built-in. As escape sequence is generally a two character-
string beginning with a \ . \c is an escape sequence. When this escape sequence is placed at the end of a string
used as an argument to echo, the command interprets the sequence as a directive to place the cursor and
prompt in the same line that displays the output:

$ echo ― Enter filename:\c‖

Enter filename:$_

There are two commonly used ones:

\t- A tab which pushes text to the right by eight character positions.
\n- newline
\a- Bell
\b-Backspace
\f-Formfed
\r-Carriage return
\v- vertical tab

Printf:
The printf command provides a method to print preformatted text similar to the printf() system
interface (C function). It's meant as successor for echo and has far more features and possibilities.
Beside other reasons, POSIX has a very good argument to recommend it: Both historical main
flavours of the echo command are mutual exclusive, they collide. A "new" command had to be
invented to solve the issue.

printf <FORMAT> <ARGUMENTS...>

printf "Surname: %s\nName: %s\n" "$SURNAME" "$FIRSTNAME"
commomly used ones for printf:
%s- String
%30s-As above but printed in a space 30 characters wide
%d- Decimal integer
%6d- printed in a space 6 characters wide
%o-Octal integer
%x- Hexadecimal integer
%f- Floating point number

who:
UNIX is a system that can be concurrently used by multiple users, and you might be interested in
knowing the people who are also using' the system like you. Use the who command

$who

Raghu pts/0 May 9 09:32

Kumar pts/1 May 9 09:32

1s: Listing Files

Your UNIX system has a large number of files that control its functioning, and users also create files
on their own. These files are organized in separate folders called directories. You can list the names of
the files available in this directory with the 1s command

$ 1s ehap*
chapOI chap02
chap03
Sometimes, just displaying a list of filenames isn't enough; you need to know more about these
files. For that to happen, 15 bas to be used with an option, -1, between the command and filenames:
$ls -l
-rw-r--r-- 1 kumar users 5609 Apr 23 09:30 chapO
-rw-r--r-- 1 kumar users 26129 May 14 18:55 chap02
1
-rw-r--r-- 1 kumar users 37385 May 15 10:30 chap03

More command:

The man command displays its output a page at a time. This is possible because it sends its output to a paper
program. Unix have more pager which has replaced pg, the original pager of unix.

more chap01

You can see the content on the screen, one page at a time. At the bottom of the screen, you will also see the
filename and percentage of the file that has been viewed.

--more—(17%)

More having couple of internal commands

1. Navigation
Irrespective of version, more uses the spacebar to scroll forward a page at a time. You can also scroll
by small and large increments ofLines or screens. To move forward one page, use

f or the spacebar
and to move back one page, use b

2. Repeating The Last Command (.) more has a repeat command, the dot (same command used
by vi), that repeats the last command you used. If you scroll forward with lOf, you can scroll
another10 pages by simply pressing a dot. This is a great convenience available in more!

3.Searching pattren
You can perform a search for a pattern with the /command followed by the string. For instance, to
look for the first wh11 e loop in your program, you'll have to enter this:

/while
4.Using more in a Pipeline
The 1s output won't fit on the screen if there are too many files, so the command has to be used like
this:
$ls | more

uname: KNOWING YOUR MACHINE'S CHARACTERISTICS

The uname command displays certain features of the operating system running on your m achine.
By default, it simply displays the name of the operating system:

$uname
Linux
. Linux systems simply show the name L1nux. Using suitable options.
The Current Release (-r) Since UNIX comes in many flavors, vendors have customized a number of
commands to behave in the way they want, and not as AT&T decreed. A UNIX command often
varies across versions so much so that you'll need to use the -r option to find out the version of your
operating system:

$uname –r
5.8

Passwd:

To change password in unix we are using this command.

$passwd
Passwd: changing password for student
Enter login passwd: *******
New password: *******
Re-enter new password: *******
Passwd(SYSTEM): passwd successfully changed for student
13.tty: KNOWIN YOUR TERMINAL
Since UNIX treats even terminals as files, it's reasonable to expect a command that
tells you the filename of the terminal you are using. It's the tty (teletype)
command, an obvious reference to the device that has now become obsolete. The
command is simple and needs no arguments:
$ tty
/dev/pts/10
The terminal filename is 10 (a file named 10) resident in the pts directory. This
directory in turn is under the /dev directory. These terminal names were seen on
a Solaris machine; your terminal names could be different (say, /dev /tty01).
You can use tty in a shell script to control the behavior of the script depending on
the terminal it is invoked from. If a program must run from only one specified
terminal, the script logic must use tty to make this decision.

stty: DISPLAYING AND SETIING TERMINAL CHARACTERISTICS

Different terminals have different characteristics, and your terminal may not
behave in the way you expect it to. For instance, command interruption may not
be possible with [Ctrl-c] on your system. Sometimes you may Liketo change the
settings to match the ones used at your previous place of work. The stty command
helps straighten these things out; it both displays and changes settings.

Changing the Settings

Whether Backspacing Should Erase Character (echoe) If you have worked on a
number of terminals, you would have noticed that backspacing over a character
sometimes removes it from sight and sometimes doesn't. This is decided by the
keyword echoe. Since it is set here (no - prefix to it), backspacing removes the
character from display.
You can use the same keyword to reverse this setting. Here you need to prefix a
- to the echoe keyword:
stty -echoe
Backspacing now doesn't remove a character from sight. This setting is inoperative on
some systems.
Entering a Passtuord through a Shell Script (echo) The echo setting has to be
manipulated to let shell programs accept a password-like string that must not be
displayed on the screen. By default, the option is turned on, but you can turn it 01T
in this way:
stty -echo Turns off keyboard input
With this setting, keyboard entry is not echoed. You should turn it off after the entry
is complete by using stty echo, which again is not displayed, but makes sure that
aU subsequent input is.
 Changing the Interrupt Key (intr) stty also sets the funcrions for some of the
keys. For instance, if you like to use [Ctrl-c] as the interrupt key instead of [Delete],
you'll have to use
stty intr \AC "and c
Here, the keyword; ntr is followed by a space, the \ (backslash) character, a A
(caret), and finally the character c. This is the way stty indicates to the system that
the interrupt character is [Ctrl-c].
When you insert control characters into a file, you'll see a A symbol apparently
prefixed to the character. For example, [Ctrl-l] is seen as Al (or "L). However,
it's actually a single character, occupying two slots on the terminal; no caret is
actually present. However, for using a control character in an stty setting, you'll
have to use a literal caret preceded by a backslash.

14. /etc/passwd

Traditional Unix systems keep user account information, including one-way

encrypted passwords, in a text file called ``/etc/passwd''. As this file is used by many
tools (such as ``ls'') to display file ownerships, etc. by matching user id #'s with
the user's names, the file needs to be world-readable. Consequentally, this can be
somewhat of a security risk.

With shadow passwords, the ``/etc/passwd'' file contains account information,

and looks like this:

smithj:x:561:561:Joe Smith:/home/smithj:/bin/bash

Each field in a passwd entry is separated with ":" colon characters, and are as
follows:

 Username, up to 8 characters. Case-sensitive, usually all lowercase

 An "x" in the password field. Passwords are stored in the ``/etc/shadow'' file.
 Numeric user id. This is assigned by the ``adduser'' script. Unix uses this
field, plus the following group field, to identify which files belong to the
user.
 Numeric group id.
 Full name of user. I'm not sure what the maximum length for this field is,
but try to keep it reasonable (under 30 characters).
 User's home directory. Usually /home/username (eg. /home/smithj). All
user's personal files, web pages, mail forwarding, etc. will be stored here.
 User's "shell account". Often set to ``/bin/bash'' to provide access to the bash
shell (my personal favorite shell).
/etc/shadow

Another method of storing account information, one that I always use, is with the
shadow password format. As with the traditional method, this method stores
account information in the /etc/passwd file in a compatible format. However, the
password is stored as a single "x" character (ie. not actually stored in this file). A
second file, called ``/etc/shadow'', contains encrypted password as well as other
information such as account or password expiration values, etc. The /etc/shadow
file is readable only by the root account and is therefore less of a security risk

The ``/etc/shadow'' file contains password and account expiration information for
users, and looks like this:

smithj:Ep6mckrOLChF.:10063:0:99999:7:::

As with the passwd file, each field in the shadow file is also separated
with ":" colon characters, and are as follows:

 Username, up to 8 characters. Case-sensitive, usually all lowercase. A

direct match to the username in the /etc/passwd file.
 Password, 13 character encrypted. A blank entry (eg. ::) indicates a
password is not required to log in (usually a bad idea), and a ``*'' entry (eg.
:*:) indicates the account has been disabled.
 The number of days (since January 1, 1970) since the password was last
changed.
 The number of days before password may be changed (0 indicates it may
be changed at any time)
 The number of days after which password must be changed (99999
indicates user can keep his or her password unchanged for many, many
years)
 The number of days to warn user of an expiring password (7 for a full
week)
 The number of days after password expires that account is disabled
 The number of days since January 1, 1970 that an account has been
disabled
 A reserved field for possible future use
Module -2
Chapter 3. The File System

Introduction
In this chapter we will look at the file system of UNIX. We also look at types of files
their significance. We then look at two ways of specifying a file viz., with absolute
pathnames and relative pathnames. A discussion on commands used with directory files
viz., cd, pwd, mkdir, rmdir and ls will be made. Finally we look at some of the important
directories contained under UNIX file system.

1. Types of files
A simple description of the UNIX system is this:
―On a UNIX system, everything is a file; if something is not a file, it is a process.‖
A UNIX system makes no difference between a file and a directory, since a directory is
just a file containing names of other files. Programs, services, texts, images, and so forth,
are all files. Input and output devices, and generally all devices, are considered to be files,
according to the system.

Most files are just files, called regular files; they contain normal data, for example text
files, executable files or programs, input for or output from a program and so on.
While it is reasonably safe to suppose that everything you encounter on a UNIX system is
a file, there are some exceptions.
Directories: files that are lists of other files.
Special files or Device Files: All devices and peripherals are represented by files. To read
or write a device, you have to perform these operations on its associated file. Most
special files are in /dev.
Links: a system to make a file or directory visible in multiple parts of the system's file
tree.
(Domain) sockets: a special file type, similar to TCP/IP sockets, providing inter−process
networking protected by the file system's access control.
Named pipes: act more or less like sockets and form a way for processes to communicate
with each other, without using network socket semantics
Ordinary (Regular) File
This is the most common file type. An ordinary file can be either a text file or a binary
file.
A text file contains only printable characters and you can view and edit them. All C and
Java program sources, shell scripts are text files. Every line of a text file is terminated
with the newline character.
A binary file, on the other hand, contains both printable and nonprintable characters that
cover the entire ASCII range. The object code and executables that you produce by
compiling C programs are binary files. Sound and video files are also binary files.

Directory File
A directory contains no data, but keeps details of the files and subdirectories that it
contains. A directory file contains one entry for every file and subdirectory that it houses.
Each entry has two components namely, the filename and a unique identification number
of the file or directory (called the inode number).
When you create or remove a file, the kernel automatically updates its corresponding
directory by adding or removing the entry (filename and inode number) associated with
the file.

Device File
All the operations on the devices are performed by reading or writing the file representing
the device. It is advantageous to treat devices as files as some of the commands used to
access an ordinary file can be used with device files as well.
Device filenames are found in a single directory structure, /dev. A device file is not really
a stream of characters. It is the attributes of the file that entirely govern the operation of
the device. The kernel identifies a device from its attributes and uses them to operate the
device.

2. Filenames in UNIX
On a UNIX system, a filename can consist of up to 255 characters. Files may or may not
have extensions and can consist of practically any ASCII character except the / and the
Null character. You are permitted to use control characters or other nonprintable
characters in a filename. However, you should avoid using these characters while naming
a file. It is recommended that only the following characters be used in filenames:
Alphabets and numerals.
The period (.), hyphen (-) and underscore (_).
UNIX imposes no restrictions on the extension. In all cases, it is the application that
imposes that restriction. Eg. A C Compiler expects C program filenames to end with .c,
Oracle requires SQL scripts to have .sql extension.
A file can have as many dots embedded in its name. A filename can also begin with or
end with a dot.
UNIX is case sensitive; cap01, Chap01 and CHAP01 are three different filenames that
can coexist in the same directory.
3. Directories and Files
A file is a set of data that has a name. The information can be an ordinary text, a user-
written computer program, results of a computation, a picture, and so on. The file name
may consist of ordinary characters, digits and special tokens like the underscore, except
the forward slash (/). It is permitted to use special tokens like the ampersand (&) or
spaces in a filename.
Unix organizes files in a tree-like hierarchical structure, with the root directory, indicated
by a forward slash (/), at the top of the tree. See the Figure below, in which part of the
hierarchy of files and directories on the computer is shown.

4. Absolute and relative paths

A path, which is the way you need to follow in the tree structure to reach a given file, can
be described as starting from the trunk of the tree (the / or root directory). In that case, the
path starts with a slash and is called an absolute path, since there can be no mistake: only
one file on the system can comply.
Paths that don't start with a slash are always relative to the current directory. In relative
paths we also use the . and .. indications for the current and the parent directory.

The HOME variable

When you log onto the system, UNIX automatically places you in a directory called the
home directory. The shell variable HOME indicates the home directory of the user.
E.g.,
$ echo $HOME
/home/kumar
What you see above is an absolute pathname, which is a sequence of directory names
starting from root (/). The subsequent slashes are used to separate the directories.

5. pwd - print working directory

At any time you can determine where you are in the file system hierarchy with the pwd,
print working directory, command,
E.g.,:
$ pwd
/home/frank/src
6. cd - change directory
You can change to a new directory with the cd, change directory, command. cd will
accept both absolute and relative path names.
Syntax
cd [directory]
Examples
cd changes to user's home directory
cd / changes directory to the system's root
cd .. goes up one directory level
cd ../.. goes up two directory levels
cd /full/path/name/from/root changes directory to absolute path named
(note the leading slash)
cd path/from/current/location changes directory to path relative to current
location (no leading slash)

7. mkdir - make a directory

You extend your home hierarchy by making sub-directories underneath it. This is done
with the mkdir, make directory, command. Again, you specify either the full or relative
path of the directory.

Examples
mkdir patch Creates a directory patch under current directory
mkdir patch dbs doc Creates three directories under current directory
mkdir pis pis/progs pis/data Creates a directory tree with pis as a directory under
the current directory and progs and data as
subdirectories under pis

Note the order of specifying arguments in example 3. The parent directory should be
specified first, followed by the subdirectories to be created under it.

The system may refuse to create a directory due to the following reasons:
1. The directory already exists.
2. There may be an ordinary file by the same name in the current directory.
3. The permissions set for the current directory don’t permit the creation of files and
directories by the user.

8. rmdir - remove directory

A directory needs to be empty before you can remove it. If it’s not, you need to remove
the files first. Also, you can’t remove a directory if it is your present working directory;
you must first change out of that directory. You cannot remove a subdirectory unless you
are placed in a directory which is hierarchically above the one you have chosen to
remove.

E.g.
rmdir patch Directory must be empty
rmdir pis pis/progs pis/data Shows error as pis is not empty. However rmdir
 silently deletes the lower level subdirectories progs
and data.

9. The PATH environment variable

Environmental variables are used to provide information to the programs you use. We
have already seen one such variable called HOME.
A command runs in UNIX by executing a disk file. When you specify a command like
date, the system will locate the associated file from a list of directories specified in the
PATH variable and then executes it. The PATH variable normally includes the current
directory also.

Whenever you enter any UNIX command, you are actually specifying the name of an
executable file located somewhere on the system. The system goes through the following
steps in order to determine which program to execute:
1. Built in commands (such as cd and history) are executed within the shell.
2. If an absolute path name (such as /bin/ls) or a relative path name (such as ./myprog),
the system executes the program from the specified directory.
3. Otherwise the PATH variable is used.

10. ls - list directory contents

The command to list your directories and files is ls. With options it can provide
information about the size, type of file, permissions, dates of file creation, change and
access.
Syntax
ls [options] [argument]
Common Options
When no argument is used, the listing will be of the current directory. There are many
very useful options for the ls command. A listing of many of them follows. When using
the command, string the desired options together preceded by "-".
-a Lists all files, including those beginning with a dot (.).
-d Lists only names of directories, not the files in the directory
-F Indicates type of entry with a trailing symbol: executables with *, directories with / and
symbolic links with @
-R Recursive list
-u Sorts filenames by last access time
-t Sorts filenames by last modification time
-i Displays inode number
-l Long listing: lists the mode, link information, owner, size, last modification (time). If the file is
a symbolic link, an arrow (-->) precedes the pathname of the linked-to file.

The mode field is given by the -l option and consists of 10 characters. The first character
is one of the following:
CHARACTER IF ENTRY IS A
d directory
- plain file
b block-type special file
 c character-type special file
l symbolic link
s socket
The next 9 characters are in 3 sets of 3 characters each. They indicate the file
access permissions: the first 3 characters refer to the permissions for the user,
the next three for the users in the Unix group assigned to the file, and the last 3
to the permissions for other users on the system.
Designations are
as follows:
r read permission
w write permission
x execute permission
- no permission
E
x
a
m
p
l
e
s
1. To list the files in a
directory:
$ ls
2. To list all files in a directory, including the hidden
(dot) files:
$ ls -a
3. To get a
long listing:
$

l
s

-
a
l

t
o
t
a
l

2
4
drwxr-sr-x 5 workshop acs 512 Jun 7 11:12 .
drwxr-xr-x 6 root sys 512 May 29 09:59 ..
-rwxr-xr-x 1 workshop acs 532 May 20 15:31 .cshrc
-rw------- 1 workshop acs 525 May 20 21:29 .emacs
-rw------- 1 workshop acs 622 May 24 12:13 .history
-rwxr-xr-x 1 workshop acs 238 May 14 09:44 .login
-rw-r--r-- 1 workshop acs 273 May 22 23:53 .plan
 -rwxr-xr-x 1 workshop acs 413 May 14 09:36 .profile
-rw------- 1 workshop acs 49 May 20 20:23 .rhosts
drwx------ 3 workshop acs 512 May 24 11:18
demofiles drwx------ 2 workshop acs 512
May 21 10:48 frank
drwx------ 3 workshop acs 512 May 24 10:59 linda

11. The UNIX File

System
The root directory has many subdirectories. The following table describes
some of the
subdirectories contained
under root.

Directory Content
/bin Common programs, shared by the system, the system administrator and the users.
Contains references to all the CPU peripheral hardware, which are represented as files with
/dev
special properties.
Most important system configuration files are in /etc, this directory contains data similar to
/etc
those in the Control Panel in Windows
/home Home directories of the common users.
/lib Library files, includes files for all kinds of programs needed by the system and the users.
/sbin Programs for use by the system and the system administrator.

Temporary space for use by the system, cleaned upon reboot, so don't use this for saving any
/tmp
work!
/usr Programs, libraries, documentation etc. for all user-related programs.
Storage for all variable files and temporary files created by users, such as log files, the mail
/var queue, the print spooler area, space for temporary storage of files downloaded from the
Internet, or to keep an image of a CD before burning it.
File Attributes

Listing File Attributes

ls command is used to obtain a list of all filenames in the current directory. The
output in UNIX lingo is often referred to as the listing. Sometimes we combine this
option with other options for displaying other attributes, or ordering the list in a different
sequence. ls look up the file’s inode to fetch its attributes. It lists seven attributes of all files
in the current directory and they are:

• File type and Permissions

• Links
• Ownership
• Group ownership
• File size
• Last Modification date and time
• File name

The file type and its permissions are associated with each file. Links indicate the
number of file names maintained by the system. This does not mean that there are so
many copies of the file. File is created by the owner. Every user is attached to a group
owner. File size in bytes is displayed. Last modification time is the next field. If you
change only the permissions or ownership of the file, the modification time remains
unchanged. In the last field, it displays the file name.

For example,

$ ls –l
total 72
-rw-r--r-- 1 kumar metal 19514 may 10 13:45 chap01
-rw-r--r-- 1 kumar metal 4174 may 10 15:01 chap02
-rw-rw-rw- 1 kumar metal 84 feb 12 12:30 dept.lst
-rw-r--r-- 1 kumar metal 9156 mar 12 1999 genie.sh
drwxr-xr-x 2 kumar metal 512 may 9 10:31 helpdir
drwxr-xr-x 2 kumar metal 512 may 9 09:57 progs

Listing Directory Attributes

ls -d will not list all subdirectories in the current directory

For example,

ls –ld helpdir progs

drwxr-xr-x 2 kumar metal 512 may 9 10:31 helpdir
drwxr-xr-x 2 kumar metal 512 may 9 09:57 progs

Directories are easily identified in the listing by the first character of the first
column, which here shows a d. The significance of the attributes of a directory differs a
good deal from an ordinary file. To see the attributes of a directory rather than the files
contained in it, use ls –ld with the directory name. Note that simply using ls –d will not
list all subdirectories in the current directory. Strange though it may seem, ls has no
option to list only directories.

File Ownership

When you create a file, you become its owner. Every owner is attached to a group
owner. Several users may belong to a single group, but the privileges of the group are set
by the owner of the file and not by the group members. When the system administrator
creates a user account, he has to assign these parameters to the user:
The user-id (UID) – both its name and numeric representation
The group-id (GID) – both its name and numeric representation

File Permissions
UNIX follows a three-tiered file protection system that determines a file’s access
rights. It is displayed in the following format:

Filetype owner (rwx) groupowner (rwx) others (rwx)

For Example:

-rwxr-xr-- 1 kumar metal 20500 may 10 19:21 chap02

rwx r-x r--

owner/user group owner others

The first group has all three permissions. The file is readable, writable and
executable by the owner of the file. The second group has a hyphen in the middle slot,
which indicates the absence of write permission by the group owner of the file. The third
group has the write and execute bits absent. This set of permissions is applicable to
others.
You can set different permissions for the three categories of users – owner, group
and others. It’s important that you understand them because a little learning here can be a
dangerous thing. Faulty file permission is a sure recipe for disaster

Changing File Permissions

A file or a directory is created with a default set of permissions, which can be

determined by umask. Let us assume that the file permission for the created file is -rw-r-- r-
-. Using chmod command, we can change the file permissions and allow the owner to
execute his file. The command can be used in two ways:

In a relative manner by specifying the changes to the current permissions

In an absolute manner by specifying the final permissions
Relative Permissions

chmod only changes the permissions specified in the command line and leaves the
other permissions unchanged. Its syntax is:

chmod category operation permission filename(s)

chmod takes an expression as its argument which contains:

user category (user, group, others)
operation to be performed (assign or remove a permission)
type of permission (read, write, execute)

Category operation permission

u - user + assign r - read
g - group - remove w - write
o - others = absolute x - execute
a - all (ugo)

Let us discuss some examples:

Initially,
-rw-r--r-- 1 kumar metal 1906 sep 23:38 xstart

chmod u+x xstart

-rwxr--r-- 1 kumar metal 1906 sep 23:38 xstart

The command assigns (+) execute (x) permission to the user (u), other permissions
remain unchanged.

chmod ugo+x xstart or

chmod a+x xstart or
chmod +x xstart

-rwxr-xr-x 1 kumar metal 1906 sep 23:38 xstart

chmod accepts multiple file names in command line

chmod u+x note note1 note3

Let initially,

-rwxr-xr-x 1 kumar metal 1906 sep 23:38 xstart

chmod go-r xstart

Then, it becomes

-rwx--x--x 1 kumar metal 1906 sep 23:38 xstart

Absolute Permissions

Here, we need not to know the current file permissions. We can set all nine
permissions explicitly. A string of three octal digits is used as an expression. The
permission can be represented by one octal digit for each category. For each category, we
add octal digits. If we represent the permissions of each category by one octal digit, this is
how the permission can be represented:

 Read permission – 4 (octal 100)

 Write permission – 2 (octal 010)
 Execute permission – 1 (octal 001)

Octal Permissions Significance

0 --- no permissions
1 --x execute only
2 -w- write only
3 -wx write and execute
4 r-- read only
5 r-x read and execute
6 rw- read and write
7 rwx read, write and execute

We have three categories and three permissions for each category, so three octal
digits can describe a file’s permissions completely. The most significant digit represents
user and the least one represents others. chmod can use this three-digit string as the
expression.