Module-5

Concurrency Control Techniques

NOSQL Databases and Big Data Storage Systems
Concurrency Control
Techniques
Introduction
● Concurrency control protocols
○ Set of rules to guarantee serializability
● Two-phase locking protocols
○ Lock data items to prevent concurrent access
● Timestamp
○ Unique identifier for each transaction
● Multiversion currency control protocols
○ Use multiple versions of a data item
● Validation or certification of a transaction
Two-Phase Locking Techniques for Concurrency Control

● Lock
○ Variable associated with a data item describing status for operations that can be applied
○ One lock for each item in the database
● Binary locks
○ Two states (values)
■ Locked (1)
● Item cannot be accessed
■ Unlocked (0)
● Item can be accessed when requested
Two-Phase Locking Techniques for Concurrency Control (cont’d.)

● Transaction requests access by issuing a lock_item(X) operation

Figure 5.1 Lock and unlock operations for binary locks

Two-Phase Locking Techniques for Concurrency Control (cont’d.)

● Lock table specifies items that have locks

● Lock manager subsystem
○ Keeps track of and controls access to locks
○ Rules enforced by lock manager module
● At most one transaction can hold the lock on an item at a given time
● Binary locking too restrictive for database items
Two-Phase Locking Techniques for Concurrency Control (cont’d.)

● Shared/exclusive or read/write locks

○ Read operations on the same item are not conflicting
○ Must have exclusive lock to write
○ Three locking operations
■ read_lock(X)
■ write_lock(X)
■ unlock(X)
Slide 21-8

Figure 5.2 Locking and

unlocking operations for
two-mode (read/write, or
shared/exclusive) locks
Two-Phase Locking Techniques for Concurrency Control (cont’d.)

● Lock conversion
○ Transaction that already holds a lock allowed to convert the lock from one state to another
● Upgrading
○ Issue a read_lock operation then a write_lock operation
● Downgrading
○ Issue a read_lock operation after a write_lock operation
Guaranteeing Serializability by Two-Phase Locking

● Two-phase locking protocol

○ All locking operations precede the first unlock operation in the transaction
○ Phases
■ Expanding (growing) phase
● New locks can be acquired but none can be released
● Lock conversion upgrades must be done during this phase
■ Shrinking phase
● Existing locks can be released but none can be acquired
● Downgrades must be done during this phase

Slide
21- 10
Figure 5.3 Transactions that
do not obey two-phase
locking (a) Two transactions
T1 and T2 (b) Results of
possible serial schedules of
T1 and T2 (c) A
nonserializable schedule S
that uses locks
Guaranteeing Serializability by Two-Phase Locking

● If every transaction in a schedule follows the two-phase locking protocol,

schedule guaranteed to be serializable
● Two-phase locking may limit the amount of concurrency that can occur in a
schedule
● Some serializable schedules will be prohibited by two-phase locking protocol

Slide
21- 12
Variations of Two-Phase Locking

● Basic 2PL
○ Technique described on previous slides
● Conservative (static) 2PL
○ Requires a transaction to lock all the items it accesses before the transaction begins
■ Predeclare read-set and write-set
○ Deadlock-free protocol
● Strict 2PL
○ Transaction does not release exclusive locks until after it commits or aborts
Variations of Two-Phase Locking (cont’d.)

● Rigorous 2PL
○ Transaction does not release any locks until after it commits or aborts
● Concurrency control subsystem responsible for generating read_lock and
write_lock requests
● Locking generally considered to have high overhead
Dealing with Deadlock and Starvation

● Deadlock
○ Occurs when each transaction T in a set is waiting for some item locked by some other
transaction T’
○ Both transactions stuck in a waiting queue

Figure 5.5 Illustrating the deadlock problem (a) A partial schedule of T1′ and T2′ that is
in a state of deadlock (b) A wait-for graph for the partial schedule in (a)
Dealing with Deadlock and Starvation (cont’d.)

● Deadlock prevention protocols

○ Every transaction locks all items it needs in advance
○ Ordering all items in the database
■ Transaction that needs several items will lock them in that order
○ Both approaches impractical
● Protocols based on a timestamp
○ Wait-die
○ Wound-wait
Dealing with Deadlock and Starvation (cont’d.)

● No waiting algorithm
○ If transaction unable to obtain a lock, immediately aborted and restarted later
● Cautious waiting algorithm
○ Deadlock-free
● Deadlock detection
○ System checks to see if a state of deadlock exists
○ Wait-for graph
Dealing with Deadlock and Starvation (cont’d.)

● Victim selection
○ Deciding which transaction to abort in case of deadlock
● Timeouts
○ If system waits longer than a predefined time, it aborts the transaction
● Starvation
○ Occurs if a transaction cannot proceed for an indefinite period of time while other transactions
continue normally
○ Solution: first-come-first-served queue
 5.2 Concurrency Control Based on Timestamp Ordering

● Timestamp
○ Unique identifier assigned by the DBMS to identify a transaction
○ Assigned in the order submitted
○ Transaction start time
● Concurrency control techniques based on timestamps do not use locks
○ Deadlocks cannot occur
 Concurrency Control Based on Timestamp Ordering
(cont’d.)
● Generating timestamps
○ Counter incremented each time its value is assigned to a transaction
○ Current date/time value of the system clock
■ Ensure no two timestamps are generated during the same tick of the clock
● General approach
○ Enforce equivalent serial order on the transactions based on their timestamps
 Concurrency Control Basedon Timestamp Ordering (cont’d.)

● Timestamp ordering (TO)

○ Allows interleaving of transaction operations
○ Must ensure timestamp order is followed for each pair of conflicting operations
● Each database item assigned two timestamp values
○ read_TS(X)
○ write_TS(X)
 Concurrency Control Based on Timestamp Ordering (cont’d.)

● Basic TO algorithm
○ If conflicting operations detected, later operation rejected by aborting transaction that issued it
○ Schedules produced guaranteed to be conflict serializable
○ Starvation may occur
● Strict TO algorithm
○ Ensures schedules are both strict and conflict serializable
 Concurrency Control Based on Timestamp Ordering
(cont’d.)

● Thomas’s write rule

○ Modification of basic TO algorithm
○ Does not enforce conflict serializability
○ Rejects fewer write operations by modifying checks for write_item(X) operation
5.3 Multiversion Concurrency Control Techniques

● Several versions of an item are kept by a system

● Some read operations that would be rejected in other techniques can be
accepted by reading an older version of the item
○ Maintains serializability
● More storage is needed
● Multiversion currency control scheme types
○ Based on timestamp ordering
○ Based on two-phase locking
○ Validation and snapshot isolation techniques
Multiversion Concurrency Control Techniques (cont’d.)
● Multiversion technique based on timestamp ordering
○ Two timestamps associated with each version are kept
■ read_TS(Xi)
■ write_TS(Xi)
Multi version Concurrency Control Techniques (cont’d.)

○ Multiversion two-phase locking using certify locks

■ Three locking modes: read, write, and certify

Figure 5.6 Lock compatibility tables (a) Lock compatibility table for read/write
locking scheme (b) Lock compatibility table for read/write/certify locking scheme
 Validation (Optimistic) Techniques and Snapshot Isolation Concurrency
Control
● Optimistic techniques
○ Also called validation or certification techniques
○ No checking is done while the transaction is executing
○ Updates not applied directly to the database until finished transaction is validated
■ All updates applied to local copies of data items
○ Validation phase checks whether any of transaction’s updates violate serializability
■ Transaction committed or aborted based on result
Concurrency Control Based on Snapshot Isolation

● Transaction sees data items based on committed values of the items in the
database snapshot
○ Does not see updates that occur after transaction starts
● Read operations do not require read locks
○ Write operations require write locks
● Temporary version store keeps track of older versions of updated items
● Variation: serializable snapshot isolation (SSI)
 5.5 Granularity of Data Items and Multiple Granularity Locking

● Size of data items known as granularity

○ Fine (small)
○ Coarse (large)
● Larger the data item size, lower the degree of concurrency permitted
○ Example: entire disk block locked
● Smaller the data item size, more locks required
○ Higher overhead
● Best item size depends on transaction type

Slide
20- 29
Multiple Granularity Level Locking
● Lock can be requested at any level

Figure 5.7 A granularity hierarchy for illustrating multiple granularity level locking
Multiple Granularity Level Locking (cont’d.)

● Intention locks are needed

○ Transaction indicates along the path from the root to the desired node, what type of lock
(shared or exclusive) it will require from one of the node’s descendants
● Intention lock types
○ Intention-shared (IS)
■ Shared locks will be requested on a descendant node
○ Intention-exclusive (IX)
■ Exclusive locks will be requested
Multiple Granularity Level Locking (cont’d.)

● Intention lock types (cont’d.)

○ Shared-intension-exclusive (SIX)
■ Current node is locked in shared mode but one or more exclusive locks will be requested
on a descendant node

Figure 21.8 Lock compatibility matrix for multiple granularity locking

Multiple Granularity Level Locking (cont’d.)
● Multiple granularity locking (MGL) protocol rules
5.6 Using Locks for Concurrency Control in Indexes

● Two-phase locking can be applied to B-tree and B+ -tree indexes

○ Nodes of an index correspond to disk pages
● Holding locks on index pages could cause transaction blocking
○ Other approaches must be used
● Conservative approach
○ Lock the root node in exclusive mode and then access the appropriate child node of the root

Slide
21- 34
Using Locks for Concurrency Control in Indexes (cont’d.)

● Optimistic approach
○ Request and hold shared locks on nodes leading to the leaf node, with exclusive lock on
the leaf
● B-link tree approach
○ Sibling nodes on the same level are linked at every level
○ Allows shared locks when requesting a page
○ Requires lock be released before accessing the child node
5.7 Other Concurrency Control Issues

● Insertion
○ When new data item is inserted, it cannot be accessed until after operation is completed
● Deletion operation on the existing data item
○ Write lock must be obtained before deletion
● Phantom problem
○ Can occur when a new record being inserted satisfies a condition that a set of records
accessed by another transaction must satisfy
○ Record causing conflict not recognized by concurrency control protocol
Other Concurrency Control Issues (cont’d.)

● Interactive transactions
○ User can input a value of a data item to a transaction T based on some value written to the
screen by transaction T′, which may not have committed
○ Solution approach: postpone output of transactions to the screen until committed
● Latches
○ Locks held for a short duration
○ Do not follow usual concurrency control protocol
5.8 Summary

● Concurrency control techniques

○ Two-phase locking
○ Timestamp-based ordering
○ Multiversion protocols
○ Snapshot isolation
● Data item granularity
● Locking protocols for indexes
● Phantom problem and interactive transaction issues
 NOSQL Databases
and Big Data Storage Systems
Introduction

● NOSQL
○ Not only SQL
○ SQL systems offer many features (not everyone will use) and restrictive
● Most NOSQL systems are distributed databases or distributed storage
systems
○ Focus on semi-structured data storage, high performance, availability, data replication, and
scalability
Introduction (cont’d.)

● NOSQL systems focus on storage of “big data”

● Typical applications that use NOSQL
○ Social media
○ Web links
○ User profiles
○ Marketing and sales
○ Posts and tweets
○ Road maps and spatial data
○ Email

Slide
24- 41
6.1 Introduction to NOSQL Systems

● BigTable
○ Google’s proprietary NOSQL system
○ Column-based or wide column store
● DynamoDB (Amazon)
○ Key-value data store
● Cassandra (Facebook)
○ Uses concepts from both key-value store and column-based systems

Slide
24- 42
Introduction to NOSQL Systems (cont’d.)

● NOSQL characteristics related to distributed databases and distributed

systems
○ Scalability
○ Availability, replication, and eventual consistency
○ Replication models
■ Master-slave
■ Master-master
○ Sharding of files
○ High performance data access
Introduction to NOSQL Systems (cont’d.)

● NOSQL characteristics related to data models and query languages

○ Schema not required
○ Less powerful query languages
○ Versioning
Introduction to NOSQL Systems (cont’d.)

● Categories of NOSQL systems

○ Document-based NOSQL systems
○ NOSQL key-value stores
○ Column-based or wide column NOSQL systems
○ Graph-based NOSQL systems
○ Hybrid NOSQL systems
○ Object databases
○ XML databases
6.2 The CAP Theorem

● Various levels of consistency among replicated data items

○ Enforcing serializabilty the strongest form of consistency
■ High overhead – can reduce read/write operation performance
● CAP theorem
○ Consistency, availability, and partition tolerance
○ Not possible to guarantee all three simultaneously
■ In distributed system with data replication
The CAP Theorem (cont’d.)

● Designer can choose two of three to guarantee

○ Weaker consistency level is often acceptable in NOSQL distributed data store
○ Guaranteeing availability and partition tolerance more important
○ Eventual consistency often adopted
6.3 Document-Based NOSQL Systems and MongoDB

● Document stores
○ Collections of similar documents
● Individual documents resemble complex objects or XML documents
○ Documents are self-describing
○ Can have different data elements
● Documents can be specified in various formats
○ XML
○ JSON
MongoDB Data Model

● Documents stored in binary JSON (BSON) format

● Individual documents stored in a collection
● Example command
○ First parameter specifies name of the collection
○ Collection options include limits on size and number of documents

● Each document in collection has unique ObjectID field called _id

MongoDB Data Model (cont’d.)

● A collection does not have a schema

○ Structure of the data fields in documents chosen based on how documents will be accessed
○ User can choose normalized or denormalized design
● Document creation using insert operation

● Document deletion using remove operation

Figure 6.1 (continues)
Example of simple documents in
MongoDB (a) Denormalized
document design with
embedded subdocuments (b)
Embedded array of document
references
Figure 6.1 (cont’d.)
Example of simple
documents in MongoDB
(c) Normalized
documents (d) Inserting
the documents in Figure
6.1(c) into their
collections
MongoDB Distributed Systems Characteristics

● Two-phase commit method

○ Used to ensure atomicity and consistency of multidocument transactions
● Replication in MongoDB
○ Concept of replica set to create multiple copies on different nodes
○ Variation of master-slave approach
○ Primary copy, secondary copy, and arbiter
■ Arbiter participates in elections to select new primary if needed
MongoDB Distributed Systems Characteristics (cont’d.)

● Replication in MongoDB (cont’d.)

○ All write operations applied to the primary copy and propagated to the secondaries
○ User can choose read preference
■ Read requests can be processed at any replica
● Sharding in MongoDB
○ Horizontal partitioning divides the documents into disjoint partitions (shards)
○ Allows adding more nodes as needed
○ Shards stored on different nodes to achieve load balancing
MongoDB Distributed Systems Characteristics (cont’d.)

● Sharding in MongoDB (cont’d.)

○ Partitioning field (shard key) must exist in every document in the collection
■ Must have an index
○ Range partitioning
■ Creates chunks by specifying a range of key values
■ Works best with range queries
○ Hash partitioning
■ Partitioning based on the hash values of each shard key
6.4 NOSQL Key-Value Stores

● Key-value stores focus on high performance, availability, and scalability

○ Can store structured, unstructured, or semi-structured data
● Key: unique identifier associated with a data item
○ Used for fast retrieval
● Value: the data item itself
○ Can be string or array of bytes
○ Application interprets the structure
● No query language
DynamoDB Overview

● DynamoDB part of Amazon’s Web Services/SDK platforms

○ Proprietary
● Table holds a collection of self-describing items
● Item consists of attribute-value pairs
○ Attribute values can be single or multi-valued
● Primary key used to locate items within a table
○ Can be single attribute or pair of attributes
Voldemort Key-Value Distributed Data Store

● Voldemort: open source key-value system similar to DynamoDB

● Voldemort features
○ Simple basic operations (get, put, and delete)
○ High-level formatted data values
○ Consistent hashing for distributing (key, value) pairs
○ Consistency and versioning
■ Concurrent writes allowed
■ Each write associated with a vector clock
Figure 6.2 Example of consistent
hashing (a) Ring having three nodes
A, B, and C, with C having greater
capacity. The h(K) values that map
to the circle points in range 1 have
their (k, v) items stored in node A,
range 2 in node B, range 3 in node C
(b) Adding a node D to the ring.
Items in range 4 are moved to the
node D from node B (range 2 is
reduced) and node C (range 3 is
reduced)
Examples of Other Key-Value Stores

● Oracle key-value store

○ Oracle NOSQL Database
● Redis key-value cache and store
○ Caches data in main memory to improve performance
○ Offers master-slave replication and high availability
○ Offers persistence by backing up cache to disk
● Apache Cassandra
○ Offers features from several NOSQL categories
○ Used by Facebook and others
24.5 Column-Based or Wide Column
NOSQL Systems
● BigTable: Google’s distributed storage system for big data
○ Used in Gmail
○ Uses Google File System for data storage and distribution
● Apache Hbase a similar, open source system
○ Uses Hadoop Distributed File System (HDFS) for data storage
○ Can also use Amazon’s Simple Storage System (S3)

Slide
24- 62
Hbase Data Model and Versioning

● Data organization concepts

○ Namespaces
○ Tables
○ Column families
○ Column qualifiers
○ Columns
○ Rows
○ Data cells
● Data is self-describing

Slide
24- 63
Hbase Data Model and Versioning (cont’d.)

● HBase stores multiple versions of data items

○ Timestamp associated with each version
● Each row in a table has a unique row key
● Table associated with one or more column families
● Column qualifiers can be dynamically specified as new table rows are created
and inserted
● Namespace is collection of tables
● Cell holds a basic data item
Figure 24.3 Examples in Hbase (a) Creating a
table called EMPLOYEE with three column
families: Name, Address, and Details (b)
Inserting some in the EMPLOYEE table;
different rows can have different self-describing
column qualifiers (Fname, Lname, Nickname,
Mname, Minit, Suffix, … for column family
Name; Job, Review, Supervisor, Salary for
column family Details). (c) Some CRUD
operations of Hbase
Hbase Crud Operations

● Provides only low-level CRUD (create, read, update, delete) operations

● Application programs implement more complex operations
● Create
○ Creates a new table and specifies one or more column families associated with the table
● Put
○ Inserts new data or new versions of existing data items
Hbase Crud Operations (cont’d.)

● Get
○ Retrieves data associated with a single row
● Scan
○ Retrieves all the rows
Hbase Storage and Distributed System Concepts

● Each Hbase table divided into several regions

○ Each region holds a range of the row keys in the table
○ Row keys must be lexicographically ordered
○ Each region has several stores
■ Column families are assigned to stores
● Regions assigned to region servers for storage
○ Master server responsible for monitoring the region servers
● Hbase uses Apache Zookeeper and HDFS
6.6 NOSQL Graph Databases and Neo4j

● Graph databases
○ Data represented as a graph
○ Collection of vertices (nodes) and edges
○ Possible to store data associated with both individual nodes and individual edges
● Neo4j
○ Open source system
○ Uses concepts of nodes and relationships
Neo4j (cont’d.)

● Nodes can have labels

○ Zero, one, or several
● Both nodes and relationships can have properties
● Each relationship has a start node, end node, and a relationship type
● Properties specified using a map pattern
● Somewhat similar to ER/EER concepts
Neo4j (cont’d.)

● Creating nodes in Neo4j

○ CREATE command
○ Part of high-level declarative query language Cypher
○ Node label can be specified when node is created
○ Properties are enclosed in curly brackets
Neo4j (cont’d.)

Figure 6.4 Examples in Neo4j using the Cypher language (a) Creating some nodes
Neo4j (cont’d.)

Figure 6.4 (cont’d.) Examples in Neo4j using the Cypher language

(b) Creating some relationships
Neo4j (cont’d.)

● Path
○ Traversal of part of the graph
○ Typically used as part of a query to specify a pattern
● Schema optional in Neo4j
● Indexing and node identifiers
○ Users can create for the collection of nodes that have a particular label
○ One or more properties can be indexed
The Cypher Query Language of Neo4j
● Cypher query made up of clauses
● Result from one clause can be the input to the next clause in the query

Slide
24- 75
Slide 24- 76

The Cypher Query Language of Neo4j (cont’d.)

Figure 24.4 (cont’d.) Examples in Neo4j using the Cypher language

(c) Basic syntax of Cypher queries
Slide 24- 77

The Cypher Query Language of Neo4j (cont’d.)

Figure 24.4 (cont’d.) Examples in

Neo4j using the Cypher language
(d) Examples of Cypher queries
Neo4j Interfaces and Distributed System Characteristics

● Enterprise edition versus community edition

○ Enterprise edition supports caching, clustering of data, and locking
● Graph visualization interface
○ Subset of nodes and edges in a database graph can be displayed as a graph
○ Used to visualize query results
● Master-slave replication
● Caching
● Logical logs
6.7 Summary

● NOSQL systems focus on storage of “big data”

● General categories
○ Document-based
○ Key-value stores
○ Column-based
○ Graph-based
○ Some systems use techniques spanning two or more categories
● Consistency paradigms
● CAP theorem
End of Module-5