DBMS (Database Management System)
DBMS (Database Management System)
ts
by all students, we can then conclude about toppers and average marks.
Va
A database management system stores data in such a way that it
becomes easier to retrieve, manipulate, and produce information.
p
Characteristics
ee
Traditionally, data was organized in file formats. DBMS was a new concept
rd
then, and all the research was done to make it overcome the deficiencies in
Pa
ts
different filtering options as required to retrieve a set of data. Traditionally it
Va
was not possible where file-processing system was used.
Multiple views − DBMS offers multiple views for different users. A user who is
in the Sales department will have a different view of database than a person
working in the Production department. This feature enables the users to have a
concentrate view of the database according to their requirements.
Security − Features like multiple views offer security to some extent where
users are unable to access data of other users and departments. DBMS offers
methods to impose constraints while entering data into the database and
retrieving the same at a later stage. DBMS offers many different levels of
security features, which enables multiple users to have different views with
different features. For example, a user in the Sales department cannot see the
data that belongs to the Purchase department. Additionally, it can also be
managed how much data of the Sales department should be displayed to the
user. Since a DBMS is not saved on the disk as traditional file systems, it is very
hard for miscreants to break the code.
Users
A typical DBMS has users with different rights and permissions who use it
for different purposes. Some users retrieve data and some back it up. The
users of a DBMS can be broadly categorized as follows −
ts
Va
p
ee
rd
administrating the database. They are responsible to look after its usage and by
whom it should be used. They create access profiles for users and apply
limitations to maintain isolation and force security. Administrators also look
after DBMS resources like system license, required tools, and other software
and hardware related maintenance.
Designers − Designers are the group of people who actually work on the
designing part of the database. They keep a close watch on what data should be
kept and in what format. They identify and design the whole set of entities,
relations, constraints, and views.
End Users − End users are those who actually reap the benefits of having a
DBMS. End users can range from simple viewers who pay attention to the logs
or market rates to sophisticated users such as business analysts.
DBMS - Architecture
The design of a DBMS depends on its architecture. It can be centralized or
decentralized or hierarchical. The architecture of a DBMS can be seen as
either single tier or multi-tier. An n-tier architecture divides the whole
system into related but independent n modules, which can be
independently modified, altered, changed, or replaced.
In 1-tier architecture, the DBMS is the only entity where the user directly
sits on the DBMS and uses it. Any changes done here will directly be done
on the DBMS itself. It does not provide handy tools for end-users. Database
designers and programmers normally prefer to use single-tier architecture.
ts
through which the DBMS can be accessed. Programmers use 2-tier
Va
architecture where they access the DBMS by means of an application. Here
the application tier is entirely independent of the database in terms of
p
operation, design, and programming.
ee
3-tier Architecture
rd
A 3-tier architecture separates its tiers from each other based on the
complexity of the users and how they use the data present in the database.
Pa
Database (Data) Tier − At this tier, the database resides along with its query
rd
processing languages. We also have the relations that define the data and their
Pa
Application (Middle) Tier − At this tier reside the application server and the
programs that access the database. For a user, this application tier presents an
abstracted view of the database. End-users are unaware of any existence of the
database beyond the application. At the other end, the database tier is not
aware of any other user beyond the application tier. Hence, the application layer
sits in the middle and acts as a mediator between the end-user and the
database.
User (Presentation) Tier − End-users operate on this tier and they know
nothing about any existence of the database beyond this layer. At this layer,
multiple views of the database can be provided by the application. All views are
generated by applications that reside in the application tier.
Multiple-tier database architecture is highly modifiable, as almost all its
components are independent and can be changed independently.
The very first data model could be flat data-models, where all the data used
are to be kept in the same plane. Earlier data models were not so scientific,
hence they were prone to introduce lots of duplication and update
anomalies.
ts
Entity-Relationship Model
Va
Entity-Relationship (ER) Model is based on the notion of real-world entities
p
and relationships among them. While formulating real-world scenario into
ee
the database model, the ER Model creates entity set, relationship set,
general attributes and constraints.
rd
ER Model is based on −
Mapping cardinalities −
o one to one
o one to many
o many to one
ts
o many to many
Relational Model
Va
The most popular data model in DBMS is the Relational Model. It is more
p
scientific a model than others. This model is based on first-order predicate
ee
Database Schema
ts
A database schema is the skeleton structure that represents the logical view
Va
of the entire database. It defines how the data is organized and how the
relations among them are associated. It formulates all the constraints that
p
are to be applied on the data.
ee
A database schema defines its entities and the relationship among them. It
rd
data and its form of storage like files, indices, etc. It defines how the data will
be stored in a secondary storage.
Logical Database Schema − This schema defines all the logical constraints
that need to be applied on the data stored. It defines tables, views, and
integrity constraints.
Database Instance
It is important that we distinguish these two terms individually. Database
schema is the skeleton of database. It is designed when the database
doesn't exist at all. Once the database is operational, it is very difficult to
make any changes to it. A database schema does not contain any data or
information.
A database instance is a state of operational database with data at any
given time. It contains a snapshot of the database. Database instances tend
to change with time. A DBMS ensures that its every instance (state) is in a
valid state, by diligently following all the validations, constraints, and
conditions that the database designers have imposed.
ts
Data Independence
Va
A database system normally contains a lot of data in addition to users’ data.
For example, it stores data about data, known as metadata, to locate and
retrieve data easily. It is rather difficult to modify or update a set of
p
needs to change over time to satisfy the requirements of the users. If the
rd
ts
All the schemas are logical, and the actual data is stored in bit format on
Va
the disk. Physical data independence is the power to change the physical
data without impacting the schema or logical data.
p
For example, in case we want to change or upgrade the storage system
ee
Entity
An entity can be a real-world object, either animate or inanimate, that can
be easily identifiable. For example, in a school database, students,
teachers, classes, and courses offered can be considered as entities. All
these entities have some attributes or properties that give them their
identity.
ts
An entity set is a collection of similar types of entities. An entity set may
Va
contain entities with attribute sharing similar values. For example, a
Students set may contain all the students of a school; likewise a Teachers
p
set may contain all the teachers of a school from all faculties. Entity sets
ee
Attributes
rd
Pa
Types of Attributes
Simple attribute − Simple attributes are atomic values, which cannot be
divided further. For example, a student's phone number is an atomic value of
10 digits.
Composite attribute − Composite attributes are made of more than one
simple attribute. For example, a student's complete name may have first_name
and last_name.
Derived attribute − Derived attributes are the attributes that do not exist in
the physical database, but their values are derived from other attributes present
in the database. For example, average_salary in a department should not be
saved directly in the database, instead it can be derived. For another example,
age can be derived from data_of_birth.
ts
email_address, etc.
Va
These attribute types can come together in a way like −
p
simple single-valued attributes
ee
Candidate Key − A minimal super key is called a candidate key. An entity set
may have more than one candidate key.
Primary Key − A primary key is one of the candidate keys chosen by the
database designer to uniquely identify the entity set.
Relationship
The association among entities is called a relationship. For example, an
employee works_at a department, a student enrolls in a course. Here,
Works_at and Enrolls are called relationships.
Relationship Set
A set of relationships of similar type is called a relationship set. Like
entities, a relationship too can have attributes. These attributes are
called descriptive attributes.
Degree of Relationship
ts
The number of participating entities in a relationship defines the degree of
the relationship.
Va
Binary = degree 2
p
Ternary = degree 3
ee
n-ary = degree
rd
Mapping Cardinalities
Cardinality defines the number of entities in one entity set, which can be
Pa
associated with the number of entities of other set via relationship set.
One-to-one − One entity from entity set A can be associated with at most one
entity of entity set B and vice versa.
One-to-many − One entity from entity set A can be associated with more than
one entities of entity set B however an entity from entity set B, can be
associated with at most one entity.
ts
Va
p
ee
rd
Pa
Many-to-one − More than one entities from entity set A can be associated with
at most one entity of entity set B, however an entity from entity set B can be
associated with more than one entity from entity set A.
Many-to-many − One entity from A can be associated with more than one
entity from B and vice versa.
ts
Va
p
ee
rd
Pa
ER Diagram Representation
Let us now learn how the ER Model is represented by means of an ER
diagram. Any object, for example, entities, attributes of an entity,
relationship sets, and attributes of relationship sets, can be represented
with the help of an ER diagram.
Entity
Entities are represented by means of rectangles. Rectangles are named with
the entity set they represent.
Attributes
ts
Attributes are the properties of entities. Attributes are represented by
Va
means of ellipses. Every ellipse represents one attribute and is directly
connected to its entity (rectangle).
p
ee
rd
Pa
If the attributes are composite, they are further divided in a tree like
structure. Every node is then connected to its attribute. That is, composite
attributes are represented by ellipses that are connected with an ellipse.
Multivalued attributes are depicted by double ellipse.
ts
Va
p
ee
rd
Pa
Many-to-one − When more than one instance of entity is associated with the
ts
relationship, it is marked as 'N:1'. The following image reflects that more than
Many-to-many − The following image reflects that more than one instance of
an entity on the left and more than one instance of an entity on the right can be
associated with the relationship. It depicts many-to-many relationship.
Participation Constraints
Total Participation − Each entity is involved in the relationship. Total
participation is represented by double lines.
Partial participation − Not all entities are involved in the relationship. Partial
participation is represented by single lines.
ts
Va
Generalization Aggregation
p
ee
entity included.
Generalization
As mentioned above, the process of generalizing entities, where the
generalized entities contain the properties of all the generalized entities, is
called generalization. In generalization, a number of entities are brought
together into one generalized entity based on their similar characteristics.
For example, pigeon, house sparrow, crow and dove can all be generalized
as Birds.
Specialization
Specialization is the opposite of generalization. In specialization, a group of
entities is divided into sub-groups based on their characteristics. Take a
group ‘Person’ for example. A person has name, date of birth, gender, etc.
ts
These properties are common in all persons, human beings. But in a
Va
company, persons can be identified as employee, employer, customer, or
vendor, based on what role they play in the company.
p
ee
rd
Pa
ts
Va
p
ee
rd
Pa
For example, the attributes of a Person class such as name, age, and
gender can be inherited by lower-level entities such as Student or Teacher.
Codd's 12 Rules
Dr Edgar F. Codd, after his extensive research on the Relational Model of
database systems, came up with twelve rules of his own, which according to
him, a database must obey in order to be regarded as a true relational
database.
These rules can be applied on any database system that manages stored
data using only its relational capabilities. This is a foundation rule, which
acts as a base for all the other rules.
ts
format.
ts
Rule Va
A database must support high-level insertion, updation, and deletion. This
must not be limited to a single row, that is, it must also support union,
p
intersection and minus operations to yield sets of data records.
ee
ts
If a system has an interface that provides access to low-level records, then
Va
the interface must not be able to subvert the system and bypass security
and integrity constraints.
p
ee
rd
Pa
Relation Data Model
Relational data model is the primary data model, which is used widely
around the world for data storage and processing. This model is simple and
it has all the properties and capabilities required to process data with
storage efficiency.
Concepts
Tables − In relational data model, relations are saved in the format of
Tables. This format stores the relation among entities. A table has rows and
columns, where rows represents records and columns represent the
attributes.
Tuple − A single row of a table, which contains a single record for that
ts
relation is called a tuple.
Va
Relation instance − A finite set of tuples in the relational database system
represents relation instance. Relation instances do not have duplicate
p
tuples.
ee
Relation key − Each row has one or more attributes, known as relation
key, which can identify the row in the relation (table) uniquely.
Constraints
Every relation has some conditions that must hold for it to be a valid
relation. These conditions are called Relational Integrity Constraints.
There are three main integrity constraints −
Key constraints
Domain constraints
in a relation with a key attribute, no two tuples can have identical values for key
attributes.
Domain Constraints
ts
Va
Attributes have specific values in real-world scenario. For example, age can
only be a positive integer. The same constraints have been tried to employ
on the attributes of a relation. Every attribute is bound to have a specific
p
range of values. For example, age cannot be less than zero and telephone
ee
Relational Algebra
Relational database systems are expected to be equipped with a query
language that can assist its users to query the database instances. There
are two kinds of query languages − relational algebra and relational
calculus.
Relational Algebra
Relational algebra is a procedural query language, which takes instances of
relations as input and yields instances of relations as output. It uses
operators to perform queries. An operator can be either unary or binary.
They accept relations as their input and yield relations as their output.
Relational algebra is performed recursively on a relation and intermediate
results are also considered relations.
Select
Project
Union
ts
Set different
Cartesian product Va
Rename
p
We will discuss all these operations in the following sections.
ee
Notation − σp(r)
For example −
σsubject = "database"(Books)
Output − Selects tuples from books where subject is 'database' and 'price'
is 450.
σsubject = "database" and price = "450" or year > "2010"(Books)
Output − Selects tuples from books where subject is 'database' and 'price'
is 450 or those books published after 2010.
For example −
ts
∏subject, author (Books)
Va
Selects and projects columns named as subject and author from the
relation Books.
p
r ∪ s = { t | t ∈ r or t ∈ s}
Notion − r U s
Output − Projects the names of the authors who have either written a book
or an article or both.
Set Difference (−)
The result of set difference operation is tuples, which are present in one
relation but are not in the second relation.
Notation − r − s
Output − Provides the name of authors who have written books but not
articles.
ts
Notation − r Χ s Va
Where r and s are relations and their output will be defined as −
p
ee
r Χ s = { q t | q ∈ r and t ∈ s}
rd
Output − Yields a relation, which shows all the books and articles written
Pa
by tutorialspoint.
Notation − ρ x (E)
Set intersection
Assignment
Natural join
Relational Calculus
In contrast to Relational Algebra, Relational Calculus is a non-procedural
query language, that is, it tells what to do but never explains how to do it.
Notation − {T | Condition}
ts
For example − Va
{ T.name | Author(T) AND T.article = 'database' }
p
Output − Returns tuples with 'name' from Author who has written article
ee
on 'database'.
rd
TRC can be quantified. We can use Existential (∃) and Universal Quantifiers
(∀).
Pa
For example −
Output − The above query will yield the same result as the previous one.
Notation −
For example −
Output − Yields Article, Page, and Subject from the relation TutorialsPoint,
where subject is database.
Just like TRC, DRC can also be written using existential and universal
quantifiers. DRC also involves relational operators.
ts
Va
p
ee
rd
Mapping Entity
An entity is a real-world object with some attributes.
ts
Create table for each entity.
Va
Entity's attributes should become fields of tables with their respective data types.
Mapping Relationship
p
ee
Mapping Process
Add the primary keys of all participating Entities as fields of table with their
respective data types.
If relationship has any attribute, add each attribute as field of table.
Declare a primary key composing all the primary keys of participating entities.
ts
Mapping Process
Va
p
Create table for weak entity set.
ee
Declare primary key of higher-level table and the primary key for lower-level
table.
SQL Overview
SQL is a programming language for Relational Databases. It is designed
over relational algebra and tuple relational calculus. SQL comes as a
package with all major distributions of RDBMS.
SQL comprises both data definition and data manipulation languages. Using
the data definition properties of SQL, one can design and modify database
schema, whereas data manipulation properties allows SQL to store and
retrieve data from database.
CREATE
Creates new databases, tables and views from RDBMS.
ts
For example − Va
Create database tutorialspoint;
p
Create table article;
ee
DROP
rd
For example−
ALTER
Modifies database schema.
For example−
SELECT/FROM/WHERE
INSERT INTO/VALUES
UPDATE/SET/WHERE
DELETE FROM/WHERE
ts
These basic constructs allow database programmers and users to enter data
Va
and information into the database and retrieve efficiently using a number of
filter options.
p
SELECT/FROM/WHERE
ee
For example −
Select author_name
From book_author
Where age > 50;
Syntax−
INSERT INTO table (column1 [, column2, column3 ... ]) VALUES (value1 [, value2, value3 ... ])
Or
For example −
UPDATE/SET/WHERE
ts
This command is used for updating or modifying the values of columns in a
Va
table (relation).
Syntax −
p
ee
UPDATE table_name SET column_name = value [, column_name = value ...] [WHERE condition]
rd
For example −
Pa
DELETE/FROM/WHERE
This command is used for removing one or more rows from a table
(relation).
Syntax −
For example −
Armstrong's Axioms
If F is a set of functional dependencies then the closure of F, denoted as F +,
ts
is the set of all functional dependencies logically implied by F. Armstrong's
Va
Axioms are a set of rules, that when applied repeatedly, generates a closure
of functional dependencies.
p
Reflexive rule − If alpha is a set of attributes and beta is_subset_of alpha,
ee
holds. That is adding attributes in dependencies, does not change the basic
dependencies.
Update anomalies − If data items are scattered and are not linked to each
other properly, then it could lead to strange situations. For example, when we
try to update one data item having its copies scattered over several places, a
few instances get updated properly while a few others are left with old values.
Such instances leave the database in an inconsistent state.
ts
Insert anomalies − We tried to insert data in a record that does not exist at
all. Va
Normalization is a method to remove all these anomalies and bring the
p
database to a consistent state.
ee
First Normal Form is defined in the definition of relations (tables) itself. This
rule defines that all the attributes in a relation must have atomic domains.
Pa
ts
following −
Va
Prime attribute − An attribute, which is a part of the prime-key, is known as a
prime attribute.
p
Non-prime attribute − An attribute, which is not a part of the prime-key, is
ee
then there should not be any proper subset Y of X, for which Y → A also
holds true.
We see here in Student_Project relation that the prime key attributes are
Stu_ID and Proj_ID. According to the rule, non-key attributes, i.e.
Stu_Name and Proj_Name must be dependent upon both and not on any of
the prime key attribute individually. But we find that Stu_Name can be
identified by Stu_ID and Proj_Name can be identified by Proj_ID
independently. This is calledpartial dependency, which is not allowed in
Second Normal Form.
ts
Va
We broke the relation in two as depicted in the above picture. So there
exists no partial dependency.
o X is a superkey or,
o A is prime attribute.
We find that in the above Student_detail relation, Stu_ID is the key and
only prime key attribute. We find that City can be identified by Stu_ID as
well as Zip itself. Neither Zip is a superkey nor is City a prime attribute.
Additionally, Stu_ID → Zip → City, so there exists transitive dependency.
To bring this relation into third normal form, we break the relation into two
relations as follows −
ts
strict terms. BCNF states that −
Va
For any non-trivial functional dependency, X → A, X must be a super-key.
and
Pa
Zip → City
ts
Theta join combines tuples from different relations provided they satisfy the
Va
theta condition. The join condition is denoted by the symbol θ.
Notation
p
ee
R1 ⋈θ R2
R1 and R2 are relations having attributes (A1, A2, .., An) and (B1, B2,..
rd
,Bn) such that the attributes don’t have anything in common, that is R1 ∩
Pa
R2 = Φ.
Student
101 Alex 10
102 Maria 11
Subjects
Class Subject
10 Math
10 English
11 Music
11 Sports
ts
Student_Detail − Va
STUDENT ⋈Student.Std = Subject.Class SUBJECT
p
Student_detail
ee
Equijoin
When Theta join uses only equality comparison operator, it is said to be
equijoin. The above example corresponds to equijoin.
Natural Join ( ⋈)
Natural join does not use any comparison operator. It does not concatenate
the way a Cartesian product does. We can perform a Natural Join only if
there is at least one common attribute that exists between two relations. In
addition, the attributes must have the same name and domain.
Courses
ts
CS01 Database Va CS
ME01 Mechanics ME
p
ee
EE01 Electronics EE
rd
Pa
HoD
Dept Head
CS Alex
ME Maya
EE Mira
Courses ⋈ HoD
Dept CID Course Head
Outer Joins
Theta Join, Equijoin, and Natural Join are called inner joins. An inner join
includes only those tuples with matching attributes and the rest are
discarded in the resulting relation. Therefore, we need to use outer joins to
ts
include all the tuples from the participating relations in the resulting
Va
relation. There are three kinds of outer joins − left outer join, right outer
join, and full outer join.
p
All the tuples from the Left relation, R, are included in the resulting relation.
rd
If there are tuples in R without any matching tuple in the Right relation S,
then the S-attributes of the resulting relation are made NULL.
Pa
Left
A B
100 Database
101 Mechanics
102 Electronics
Right
A B
100 Alex
102 Maya
104 Mira
Courses HoD
ts
A B Va C D
Courses HoD
A B C D
100 Database 100 Alex
Courses HoD
ts
A B
Va C D
p
100 Database 100 Alex
ee
ts
Va
Primary Storage − The memory storage that is directly accessible to the CPU
comes under this category. CPU's internal memory (registers), fast memory
p
ee
(cache), and main memory (RAM) are directly accessible to the CPU, as they
are all placed on the motherboard or CPU chipset. This storage is typically very
rd
small, ultra-fast, and volatile. Primary storage requires continuous power supply
in order to maintain its state. In case of a power failure, all its data is lost.
Pa
Secondary Storage − Secondary storage devices are used to store data for
future use or as backup. Secondary storage includes memory devices that are
not a part of the CPU chipset or motherboard, for example, magnetic disks,
optical disks (DVD, CD, etc.), hard disks, flash drives, and magnetic tapes.
The memory with the fastest access is the costliest one. Larger storage
devices offer slow speed and they are less expensive, however they can
store huge volumes of data as compared to CPU registers or cache memory.
Magnetic Disks
ts
Hard disk drives are the most common secondary storage devices in
present computer systems. These are called magnetic disks because they
Va
use the concept of magnetization to store information. Hard disks consist of
metal disks coated with magnetizable material. These disks are placed
p
vertically on a spindle. A read/write head moves in between the disks and is
ee
hard disk plate has many concentric circles on it, called tracks. Every track
is further divided into sectors. A sector on a hard disk typically stores 512
bytes of data.
RAID 1
ts
RAID 1 uses mirroring techniques. When data is sent to a RAID controller, it
sends a copy of data to all the disks in the array. RAID level 1 is also
Va
calledmirroring and provides 100% redundancy in case of a failure.
p
ee
rd
Pa
RAID 2
RAID 2 records Error Correction Code using Hamming distance for its data,
striped on different disks. Like level 0, each data bit in a word is recorded
on a separate disk and ECC codes of the data words are stored on a
different set disks. Due to its complex structure and high cost, RAID 2 is not
commercially available.
RAID 3
RAID 3 stripes the data onto multiple disks. The parity bit generated for
data word is stored on a different disk. This technique makes it to overcome
single disk failures.
RAID 4
In this level, an entire block of data is written onto data disks and then the
ts
parity is generated and stored on a different disk. Note that level 3 uses
byte-level striping, whereas level 4 uses block-level striping. Both level 3
Va
and level 4 require at least three disks to implement RAID.
p
ee
rd
Pa
RAID 5
RAID 5 writes whole data blocks onto different disks, but the parity bits
generated for data block stripe are distributed among all the data disks
rather than storing them on a different dedicated disk.
RAID 6
RAID 6 is an extension of level 5. In this level, two independent parities are
generated and stored in distributed fashion among multiple disks. Two
parities provide additional fault tolerance. This level requires at least four
disk drives to implement RAID.
ts
Va
p
ee
rd
Pa
DBMS - File Structure
Relative data and information is stored collectively in file formats. A file is a
sequence of records stored in binary format. A disk drive is formatted into
several blocks that can store records. File records are mapped onto those
disk blocks.
File Organization
File Organization defines how file records are mapped onto disk blocks. We
have four types of File Organization to organize file records −
ts
Va
p
ee
rd
Pa
ts
this mechanism, related records from one or more relations are kept in the
Va
same disk block, that is, the ordering of records is not based on primary
key or search key.
p
File Operations
ee
Update Operations
Pa
Retrieval Operations
Open − A file can be opened in one of the two modes, read mode orwrite
mode. In read mode, the operating system does not allow anyone to alter data.
In other words, data is read only. Files opened in read mode can be shared
among several entities. Write mode allows data modification. Files opened in
write mode can be read but cannot be shared.
Locate − Every file has a file pointer, which tells the current position where the
data is to be read or written. This pointer can be adjusted accordingly. Using
find (seek) operation, it can be moved forward or backward.
Read − By default, when files are opened in read mode, the file pointer points
to the beginning of the file. There are options where the user can tell the
operating system where to locate the file pointer at the time of opening a file.
The very next data to the file pointer is read.
Write − User can select to open a file in write mode, which enables them to edit
its contents. It can be deletion, insertion, or modification. The file pointer can
be located at the time of opening or can be dynamically changed if the
operating system allows to do so.
Close − This is the most important operation from the operating system’s point
ts
of view. When a request to close a file is generated, the operating system
o
Va
removes all the locks (if in shared mode),
o saves the data (if altered) to the secondary storage media, and
p
ee
o releases all the buffers and file handlers associated with the file.
The organization of data inside a file plays a major role here. The process to
rd
locate the file pointer to a desired record inside a file various based on
whether the records are arranged sequentially or clustered.
Pa
DBMS - Indexing
We know that data is stored in the form of records. Every record has a key
field, which helps it to be recognized uniquely.
Primary Index − Primary index is defined on an ordered data file. The data file
is ordered on a key field. The key field is generally the primary key of the
relation.
ts
Secondary Index − Secondary index may be generated from a field which is a
Va
candidate key and has a unique value in every record, or a non-key with
duplicate values.
p
ee
Dense Index
Sparse Index
Dense Index
In dense index, there is an index record for every search key value in the
database. This makes searching faster but requires more space to store
index records itself. Index records contain search key value and a pointer to
the actual record on the disk.
Sparse Index
In sparse index, index records are not created for every search key. An
index record here contains a search key and an actual pointer to the data
on the disk. To search a record, we first proceed by index record and reach
at the actual location of the data. If the data we are looking for is not where
we directly reach by following the index, then the system starts sequential
search until the desired data is found.
ts
Va
p
ee
rd
Multilevel Index
Pa
Multi-level Index helps in breaking down the index into several smaller
indices in order to make the outermost level so small that it can be saved in
a single disk block, which can easily be accommodated anywhere in the
main memory.
B+ Tree
A B+ tree is a balanced binary search tree that follows a multi-level index
format. The leaf nodes of a B+ tree denote actual data pointers. B+ tree
ensures that all leaf nodes remain at the same height, thus balanced.
Additionally, the leaf nodes are linked using a link list; therefore, a B + tree
can support random access as well as sequential access.
Structure of B+ Tree
Every leaf node is at equal distance from the root node. A B + tree is of the
ordern where n is fixed for every B+ tree.
Internal nodes −
Internal (non-leaf) nodes contain at least ⌈n/2⌉ pointers, except the root node.
ts
At most, an internal node can contain n pointers.
Leaf nodes − Va
Leaf nodes contain at least ⌈n/2⌉ record pointers and ⌈n/2⌉ key values.
p
At most, a leaf node can contain n record pointers and n key values.
ee
Every leaf node contains one block pointer P to point to next leaf node and forms
a linked list.
rd
B+ Tree Insertion
Pa
B+ trees are filled from bottom and each entry is done at the leaf node.
o Partition at i = ⌊(m+1)/2⌋.
B+ Tree Deletion
B+ tree entries are deleted at the leaf nodes.
o If it is an internal node, delete and replace with the entry from the left
position.
ts
o If underflow occurs, distribute the entries from the nodes left to it.
Va
If distribution is not possible from left, then
p
o Distribute from the nodes right to it.
ee
Hash Organization
Bucket − A hash file stores data in bucket format. Bucket is considered a unit of
storage. A bucket typically stores one complete disk block, which in turn can
ts
store one or more records.
Va
Hash Function − A hash function, h, is a mapping function that maps all the
set of search-keys K to the address where actual records are placed. It is a
p
function from search keys to bucket addresses.
ee
Static Hashing
rd
used, then it shall generate only 5 values. The output address shall always
be same for that function. The number of buckets provided remains
unchanged at all times.
ts
Va
Operation
p
ee
Insertion − When a record is required to be entered using static hash, the hash
function h computes the bucket address for search key K, where the record will
rd
be stored.
Pa
Search − When a record needs to be retrieved, the same hash function can be
used to retrieve the address of the bucket where the data is stored.
Bucket Overflow
The condition of bucket-overflow is known as collision. This is a fatal state
for any static hash function. In this case, overflow chaining can be used.
Overflow Chaining − When buckets are full, a new bucket is allocated for the
same hash result and is linked after the previous one. This mechanism is
called Closed Hashing.
Linear Probing − When a hash function generates an address at which data is
already stored, the next free bucket is allocated to it. This mechanism is
called Open Hashing.
ts
Va
p
ee
rd
Pa
Dynamic Hashing
The problem with static hashing is that it does not expand or shrink
dynamically as the size of the database grows or shrinks. Dynamic hashing
provides a mechanism in which data buckets are added and removed
dynamically and on-demand. Dynamic hashing is also known as extended
hashing.
Organization
Pa
The prefix of an entire hash value is taken as a hash index. Only a portion
of the hash value is used for computing bucket addresses. Every hash index
has a depth value to signify how many bits are used for computing a hash
function. These bits can address 2n buckets. When all these bits are
consumed − that is, when all the buckets are full − then the depth value is
increased linearly and twice the buckets are allocated.
Operation
Querying − Look at the depth value of the hash index and use those bits to
compute the bucket address.
Deletion − Perform a query to locate the desired data and delete the same.
Insertion − Compute the address of the bucket
o Else
o If all the buckets are full, perform the remedies of static hashing.
Hashing is not favorable when the data is organized in some ordering and
the queries require a range of data. When data is discrete and random,
hash performs the best.
ts
Hashing algorithms have high complexity than indexing. All hash operations
are done in constant time. Va
p
ee
rd
Pa
DBMS - Transaction
A transaction can be defined as a group of tasks. A single task is the
minimum processing unit which cannot be divided further.
A’s Account
Open_Account(A)
Old_Balance = A.balance
New_Balance = Old_Balance - 500
A.balance = New_Balance
ts
Close_Account(A)
B’s Account
Va
Open_Account(B)
p
Old_Balance = B.balance
ee
Close_Account(B)
Pa
ACID Properties
A transaction is a very small unit of a program and it may contain several
lowlevel tasks. A transaction in a database system must
maintain Atomicity,Consistency, Isolation, and Durability − commonly
known as ACID properties − in order to ensure accuracy, completeness, and
data integrity.
Durability − The database should be durable enough to hold all its latest
updates even if the system fails or restarts. If a transaction updates a chunk of
data in a database and commits, then the database will hold the modified data.
If a transaction commits but the system fails before the data could be written
on to the disk, then that data will be updated once the system springs back into
action.
Isolation − In a database system where more than one transaction are being
ts
executed simultaneously and in parallel, the property of isolation states that all
the transactions will be carried out and executed as if it is the only transaction
Va
in the system. No transaction will affect the existence of any other transaction.
Serializability
p
ee
Equivalence Schedules
An equivalence schedule can be of the following types −
Result Equivalence
If two schedules produce the same result after execution, they are said to
ts
be result equivalent. They may yield the same result for some value and
Va
different results for another set of values. That's why this equivalence is not
generally considered significant.
View Equivalence
p
ee
For example −
Pa
If T reads the initial data in S1, then it also reads the initial data in S2.
If T reads the value written by J in S1, then it also reads the value written by J
in S2.
If T performs the final write on the data value in S1, then it also performs the
final write on the data value in S2.
Conflict Equivalence
Two schedules would be conflicting if they have the following properties −
States of Transactions
A transaction in a database can be in one of the following states −
ts
Va
p
ee
rd
Pa
Active − In this state, the transaction is being executed. This is the initial state
of every transaction.
ts
Va
p
ee
rd
Pa
DBMS - Concurrency Control
In a multiprogramming environment where multiple transactions can be
executed simultaneously, it is highly important to control the concurrency of
transactions. We have concurrency control protocols to ensure atomicity,
isolation, and serializability of concurrent transactions. Concurrency control
protocols can be broadly divided into two categories −
Lock-based Protocols
Database systems equipped with lock-based protocols use a mechanism by
which any transaction cannot read or write data until it acquires an
ts
appropriate lock on it. Locks are of two kinds −
Va
Binary Locks − A lock on a data item can be in two states; it is either locked or
unlocked.
p
Shared/exclusive − This type of locking mechanism differentiates the locks
ee
the same data item would lead the database into an inconsistent state. Read
Pa
ts
permission for the locks it requires. The second part is where the
Va
transaction acquires all the locks. As soon as the transaction releases its
first lock, the third phase starts. In this phase, the transaction cannot
demand any new locks; it only releases the acquired locks.
p
ee
rd
Pa
Two-phase locking has two phases, one is growing, where all the locks are
being acquired by the transaction; and the second phase is shrinking,
where the locks held by the transaction are being released.
ts
Va
Strict-2PL does not have cascading abort as 2PL does.
Timestamp-based Protocols
p
timestamp.
Pa
Every transaction has a timestamp associated with it, and the ordering is
determined by the age of the transaction. A transaction created at 0002
clock time would be older than all other transactions that come after it. For
example, any transaction 'y' entering the system at 0004 is two seconds
younger and the priority would be given to the older one.
In addition, every data item is given the latest read and write-timestamp.
This lets the system know when the last ‘read and write’ operation was
performed on the data item.
Timestamp Ordering Protocol
The timestamp-ordering protocol ensures serializability among transactions
in their conflicting read and write operations. This is the responsibility of the
protocol system that the conflicting pair of tasks should be executed
according to the timestamp values of the transactions.
ts
o
Operation rejected. Va
o If TS(Ti) >= W-timestamp(X)
Operation executed.
p
ee
Operation rejected.
ts
restarted.
Va
Deadlock Prevention
p
To prevent any deadlock situation in the system, the DBMS aggressively
ee
inspects all the operations, where transactions are about to execute. The
DBMS inspects the operations and analyzes if they can create a deadlock
rd
Wait-Die Scheme
In this scheme, if a transaction requests to lock a resource (data item),
which is already held with a conflicting lock by another transaction, then
one of the two possibilities may occur −
If TS(Ti) < TS(Tj) − that is Ti, which is requesting a conflicting lock, is older than
Tj − then Ti is allowed to wait until the data-item is available.
Wound-Wait Scheme
In this scheme, if a transaction requests to lock a resource (data item),
which is already held with conflicting lock by some another transaction, one
of the two possibilities may occur −
If TS(Ti) < TS(Tj), then Ti forces Tj to be rolled back − that is Ti wounds Tj. Tj is
restarted later with a random delay but with the same timestamp.
If TS(Ti) > TS(Tj), then Ti is forced to wait until the resource is available.
This scheme, allows the younger transaction to wait; but when an older
transaction requests an item held by a younger one, the older transaction
forces the younger one to abort and release the item.
ts
In both the cases, the transaction that enters the system at a later stage is
Va
aborted.
Deadlock Avoidance
p
ee
advance. Methods like "wait-for graph" are available but they are suitable
Pa
for only those systems where transactions are lightweight having fewer
instances of resource. In a bulky system, deadlock prevention techniques
may work well.
Wait-for Graph
This is a simple method available to track if any deadlock situation may
arise. For each transaction entering into the system, a node is created.
When a transaction Ti requests for a lock on an item, say X, which is held by
some other transaction Tj, a directed edge is created from Ti to Tj. If
Tj releases item X, the edge between them is dropped and T i locks the data
item.
The system maintains this wait-for graph for every transaction waiting for
some data items held by others. The system keeps checking if there's any
cycle in the graph.
ts
Here, we can use any of the two following approaches −
Va
First, do not allow any request for an item, which is already locked by another
p
transaction. This is not always feasible and may cause starvation, where a
ee
transaction indefinitely waits for a data item and can never acquire it.
rd
The second option is to roll back one of the transactions. It is not always feasible
to roll back the younger transaction, as it may be important than the older one.
Pa
ts
A state of active database in
Va the volatile memory can be
periodicallydumped onto a stable storage, which may also contain logs and
active transactions and buffer blocks.
p
<dump> can be marked on a log file, whenever the database contents are
ee
Recovery
Pa
When the system recovers from a failure, it can restore the latest dump.
It can recover the system by consulting undo-redo lists to restore the state of all
transactions up to the last checkpoint.
Remote Backup
ts
Remote backup provides a sense of security in case the primary location
Va
where the database is located gets destroyed. Remote backup can be offline
or real-time or online. In case it is offline, it is maintained manually.
p
ee
rd
Pa
Online backup systems are more real-time and lifesavers for database
administrators and investors. An online backup system is a mechanism
where every bit of the real-time data is backed up simultaneously at two
distant places. One of them is directly connected to the system and the
other one is kept at a remote place as backup.
As soon as the primary database storage fails, the backup system senses
the failure and switches the user system to the remote storage. Sometimes
this is so instant that the users can’t even realize a failure.
ts
executed every second. The durability and robustness of a DBMS depends
on its complex architecture and its underlying hardware and system
Va
software. If it fails or crashes amid transactions, it is expected that the
system would follow some sort of algorithm or techniques to recover lost
p
data.
ee
Failure Classification
rd
To see where the problem has occurred, we generalize a failure into various
Pa
categories, as follows −
Transaction failure
A transaction has to abort when it fails to execute or when it reaches a
point from where it can’t go any further. This is called transaction failure
where only a few transactions or processes are hurt.
System Crash
There are problems − external to the system − that may cause the system
to stop abruptly and cause the system to crash. For example, interruptions
in power supply may cause the failure of underlying hardware or software
failure.
Disk Failure
In early days of technology evolution, it was a common problem where
hard-disk drives or storage drives used to fail frequently.
ts
Disk failures include formation of bad sectors, unreachability to the disk,
Va
disk head crash or any other failure, which destroys all or a part of disk
storage.
p
Storage Structure
ee
It should check the states of all the transactions, which were being executed.
A transaction may be in the middle of some operation; the DBMS must ensure
the atomicity of the transaction in this case.
ts
There are two types of techniques, which can help a DBMS in recovering as
Va
well as maintaining the atomicity of a transaction −
Maintaining the logs of each transaction, and writing them onto some stable
p
storage before actually modifying the database.
ee
Maintaining shadow paging, where the changes are done on a volatile memory,
rd
Log-based Recovery
Log is a sequence of records, which maintains the records of actions
performed by a transaction. It is important that the logs are written prior to
the actual modification and stored on a stable storage media, which is
failsafe.
When a transaction enters the system and starts execution, it writes a log about
it.
<Tn, Start>
When the transaction modifies an item X, it write logs as follows −
<Tn, commit>
ts
operation.
Va
Recovery with Concurrent Transactions
When more than one transaction are being executed in parallel, the logs are
p
interleaved. At the time of recovery, it would become hard for the recovery
ee
system to backtrack all logs, and then start recovering. To ease this
rd
Checkpoint
Keeping and maintaining logs in real time and in real environment may fill
out all the memory space available in the system. As time passes, the log
file may grow too big to be handled at all. Checkpoint is a mechanism
where all the previous logs are removed from the system and stored
permanently in a storage disk. Checkpoint declares a point before which the
DBMS was in consistent state, and all the transactions were committed.
Recovery
When a system with concurrent transactions crashes and recovers, it
behaves in the following manner −
The recovery system reads the logs backwards from the end to the last
checkpoint.
ts
If the recovery system sees a log with <Tn, Start> and <Tn, Commit> or just
Va
<Tn, Commit>, it puts the transaction in the redo-list.
If the recovery system sees a log with <Tn, Start> but no commit or abort log
p
All the transactions in the undo-list are then undone and their logs are
rd
removed. All the transactions in the redo-list and their previous logs are
Pa