Understanding How CQL3 Maps to Cassandra's Internal Data Structure

The CQL3/Cassandra
Mapping
John Berryman
OpenSource Connections


Outline
•
•
•
•
•
•
•

What Problem does CQL Solve?
The Cassandra Data Model
Pain Points of “Old” Cassandra
Introducing CQL
Understanding the CQL/Cassandra Mapping
CQL for Sets, Lists, and Maps
Putting it All Together


• The Awesomeness that is Cassandra:
o
o
o
o
o
o

Distributed columnar data store
No single point of failure
Optimized for availability (though “Tunably” consistent)
Optimized for writes
Easily maintainable
Almost infinitely scalable

.


o
o
o
o
o
o

Easily maintainable

• Cassandra’s usability challenges
o NoSQL – “Where are my JOINS? No Schema? De-normalize!?”
o BigTable – “Tables with millions of columns!?”

.


o
o
o
o
o
o

Easily maintainable

• Cassandra’s usability challenges
o NoSQL – “Where are my JOINS? No Schema? De-normalize!?”
o BigTable – “Tables with millions of columns!?”

• CQL saves the day!
o A best-practices interface to Cassandra
o Uses familiar SQL-like language

C* Data Model
Keyspace


C* Data Model
Keyspace
Column Family

Column Family


C* Data Model
Row Key


C* Data Model
Row Key

Column
Column Name

Column Value
(or Tombstone)
Timestamp
Time-to-live


C* Data Model
Row Key

Column
Column Name

Column Value
(or Tombstone)
Timestamp
Time-to-live

● Row Key, Column Name, Column
Value have types
● Column Name has comparator
● RowKey has partitioner
● Rows can have any number of
columns - even in same column family
● Rows can have many columns
● Column Values can be omitted
● Time-to-live is useful!
● Tombstones

C* Data Model: Writes

Mem
Table

CommitLog
Row
Cache

Bloom
Filter

● Insert into
MemTable
● Dump to
CommitLog
● No read
● Very Fast!
● Blocks on CPU
before O/I!
Key
Cache
Key
Cache
Key
Cache
Key
Cache

SSTable
SSTable
SSTable
SSTable


C* Data Model:
Reads
Mem
Table

CommitLog
Row
Cache

Bloom
Filter

● Get values from Memtable
● Get values from row
cache if present
● Otherwise check bloom
filter to find appropriate
SSTables
● Check Key Cache for fast
SSTable Search
● Get values from SSTables
● Repopulate Row Cache
● Super Fast Col. retrieval
● Fast row slicing
Key
Cache
Key
Cache
Key
Cache
Key
Cache

SSTable
SSTable
SSTable
SSTable


Cassandra Pain Points
• Twitter Example
• My tweets
o SET tweets[JnBrymn][2013-07-19 T 09:20] = “Wonderful
morning. This coffee is great.”
o SET tweets[JnBrymn][2013-07-19 T 09:21] = “Oops, smoke is
coming out of the SQL server!”
o SET tweets[JnBrymn][2013-07-19 T 09:51] = “Now my coffee is
cold :-(”

• Get John’s tweets
o GET tweets[JnBrymn]

(output is as expected)


• Twitter Example
• My tweets
o SET tweets[JnBrymn][2013-07-19 T 09:20] = “Wonderful
morning. This coffee is great.”
o SET tweets[JnBrymn][2013-07-19 T 09:21] = “Oops, smoke is
coming out of the SQL server!”
o SET tweets[JnBrymn][2013-07-19 T 09:51] = “Now my coffee is
cold :-(”

• Get John’s tweets
o GET tweets[JnBrymn]

(output is as expected)

• Pain-point: schema-less means that you have to
read code to understand data model

• My timeline (other’s tweets)
• More complicated – must store corresponding user
names
• Bad Option 1: keep multiple column families
o SET timeline_from[JnBrymn][2013-07-19 T 09:20] =
“softwaredoug”
o SET timeline_text[JnBrymn][2013-07-19 T 09:20] = “Hey John I
posted on reddit, upvote me!”

• Get John’s timeline
o GET timeline_from[JnBrymn]
o GET timeline_text[JnBrymn]


• My timeline (other’s tweets)
• More complicated – must store corresponding user
names
• Bad Option 1: keep multiple column families
o SET timeline_from[JnBrymn][2013-07-19 T 09:20] =
“softwaredoug”
o SET timeline_text[JnBrymn][2013-07-19 T 09:20] = “Hey John I

o GET timeline_from[JnBrymn]
o GET timeline_text[JnBrymn]

• Pain-point: Multiple queries required.

• My timeline
• Bad Option 2: shove into single column value
o SET timeline[JnBrymn][2013-07-19 T 09:20] =
{from:”softwaredoug”, text: “Hey John I posted on reddit, upvote
me!”

o GET timeline[JnBrymn] (…not too bad.)


• My timeline
• Bad Option 2: shove into single column value
o SET timeline[JnBrymn][2013-07-19 T 09:20] =
{from:”softwaredoug”, text: “Hey John I posted on reddit, upvote
me!”

o GET timeline[JnBrymn] (…not too bad.)

• Pain-point: Updates require a read-then-modify


• My timeline
• Best Option: composite column names
o SET timeline[JnBrymn][2013-07-19 T 09:20|from] =
”softwaredoug”
o SET timeline[JnBrymn][2013-07-19 T 09:20|text] = “Hey John, I

o GET timeline[JnBrymn] (extract from and text in client)

• Resolves prior pain points! Scales well!


• My timeline
• Best Option: composite column names
o SET timeline[JnBrymn][2013-07-19 T 09:20|from] =
”softwaredoug”
o SET timeline[JnBrymn][2013-07-19 T 09:20|text] = “Hey John, I

o GET timeline[JnBrymn] (extract from and text in client)

• Resolves prior pain points! Scales well!
• Pain-point: Even more code reading to understand
data model!


• Justin Bieber’s timeline (e.g. many tweets)
• Previous solution fails if number of columns > 2Billion
• Best Option: composite row names
o SET timeline[bieber|2013-07][19 T 09:20|from] = ”softwaredoug”
o SET timeline[bieber|2013-07][19 T 09:20|text] = “Justin Bieber,
you complete me.”

• Get Justin’s timeline
o GET timeline[bieber|2013-07] (get other months too)


• Justin Bieber’s timeline (e.g. many tweets)
• Previous solution fails if number of columns > 2Billion
• Best Option: composite row names
o SET timeline[bieber|2013-07][19 T 09:20|from] = ”softwaredoug”
o SET timeline[bieber|2013-07][19 T 09:20|text] = “Justin Bieber,
you complete me.”

• Get Justin’s timeline
o GET timeline[bieber|2013-07] (get other months too)

• Pain-point: Even more code reading to understand
data model!


Introducing CQL
• CQL is a reintroduction of schema so that you don’t
have to read code to understand the data model.
• CQL creates a common language so that details of
the data model can be easily communicated.
• CQL is a best-practices Cassandra interface and
hides the messy details.


Introducing CQL
• CQL is a reintroduction of schema so that you don’t
have to read code to understand the data model.
• CQL creates a common language so that details of
the data model can be easily communicated.
• CQL is a best-practices Cassandra interface and
hides the messy details.

Let’s see it!

Introducing CQL
CREATE TABLE users (
id timeuuid PRIMARY KEY,
lastname varchar,
firstname varchar,
dateOfBirth timestamp );


Introducing CQL
lastname varchar,
firstname varchar,
INSERT INTO users (id,lastname, firstname, dateofbirth)
VALUES (now(),'Berryman',’John','1975-09-15');


Introducing CQL
lastname varchar,
firstname varchar,
VALUES (now(),’Berryman’,’John’,’1975-09-15’);
UPDATE users SET firstname = ’John’
WHERE id = f74c0b20-0862-11e3-8cf6-b74c10b01fc6;


Introducing CQL
lastname varchar,
firstname varchar,
VALUES (now(),'Berryman',’John','1975-09-15');
UPDATE users SET firstname = 'John’
WHERE id = f74c0b20-0862-11e3-8cf6-b74c10b01fc6;
SELECT dateofbirth,firstname,lastname FROM users ;
dateofbirth
| firstname | lastname
--------------------------+-----------+---------1975-09-15 00:00:00-0400 |
John | Berryman

Introducing CQL
“Hey sweet! It’s exactly the same as MySQL!”


Introducing CQL
Hold your horses. There are some
important differences.


Introducing CQL
“Wait? What happened to the
Cassandra’s wide rows?”


Introducing CQL
There’s still there. Understanding
the mapping is crucial!


Introducing CQL
There’s still there. Understanding
the mapping is crucial!

Remember this:

•Cassandra finds rows fast
•Cassandra scans columns fast
•Cassandra does not scan rows

The CQL/Cassandra Mapping
CREATE TABLE employees (
name text PRIMARY KEY,
age int,
role text
);


age int,
role text
);

name | age | role
-----+-----+----john | 37 | dev
eric | 38 | ceo


age int,
role text
);

age
john

role

37

dev

name | age | role
-----+-----+----john | 37 | dev
eric | 38 | ceo

age
eric

role

38

ceo


company text,
name text,
age int,
role text,
PRIMARY KEY (company,name)
);


company text,
name text,
age int,
role text,
);

company | name | age | role
--------+------+-----+----OSC | eric | 38 | ceo
OSC | john | 37 | dev
RKG | anya | 29 | lead
RKG | ben | 27 | dev
RKG | chad | 35 | ops


CREATE TABLE example (
A text,
B text,
C text,
D text,
E text,
F text,
PRIMARY KEY ((A,B),C,D)
);


A text,
B text,
C text,
D text,
E text,
F text,
);

A|B|C| D|E|F
--+---+---+---+---+--a|b|c|d|e|f
a|b|c|g|h|i
a|b|j|k|l|m
a|n|o|p|q|r
s|t|u|v|w|x


A text,
B text,
C text,
D text,
E text,
F text,
);
c:d:E
c:d:F
a:b

f

o:p:E
a:n

e

r

c:g:E

c:g:F

j:k:E

j:k:F

h

i

l

m

u:v:E

u:v:F

w

x

o:p:F

q

A|B|C| D|E|F
--+---+---+---+---+--a|b|c|d|e|f
a|b|c|g|h|i
a|b|j|k|l|m
a|n|o|p|q|r
s|t|u|v|w|x

s:t


• Collection Semantics
o Sets hold list of unique elements
o Lists hold ordered, possibly repeating elements
o Maps hold a list of key-value pairs

• Uses same old Cassandra data structure



• Declaring
CREATE TABLE mytable(
X text,
Y text,
myset set<text>,
mylist list<int>,
mymap map<text, text>,
PRIMARY KEY (X,Y)
);


• Declaring
X text,
Y text,
myset set<text>,
mylist list<int>,
mymap map<text, text>,
PRIMARY KEY (X,Y)
);

Collection fields
can not be used
in primary keys

• Inserting
INSERT INTO mytable (row, myset)
VALUES (123, { ‘apple’, ‘banana’});


• Inserting
INSERT INTO mytable (row, mylist)
VALUES (123, [‘apple’,’banana’,’apple’]);


• Inserting
INSERT INTO mytable (row, mylist)
VALUES (123, [‘apple’,’banana’,’apple’]);
INSERT INTO mytable (row, mymap)
VALUES (123, {1:’apple’,2:’banana’})


• Updating
UPDATE mytable SET myset = myset + {‘apple’,‘banana’}
WHERE row = 123;
UPDATE mytable SET myset = myset - { ‘apple’ }
WHERE row = 123;


• Updating
WHERE row = 123;
WHERE row = 123;
UPDATE mytable SET mylist = mylist + [‘apple’,‘banana’]
WHERE row = 123;
UPDATE mytable SET mylist = [‘banana’] + mylist
WHERE row = 123;


• Updating
WHERE row = 123;
WHERE row = 123;
UPDATE mytable SET mylist = mylist + [‘apple’,‘banana’]
WHERE row = 123;
UPDATE mytable SET mylist = [‘banana’] + mylist
WHERE row = 123;
UPDATE mytable SET mymap[‘fruit’] = ‘apple’
WHERE row = 123
UPDATE mytable SET mymap = mymap + { ‘fruit’:‘apple’}
WHERE row = 123

SETS
X text,
Y text,
myset set<int>,
PRIMARY KEY (X,Y)
);


SETS
X text,
Y text,
myset set<int>,
PRIMARY KEY (X,Y)
);

X | Y | myset
---+---+-----------a | b | {1,2}
a | c | {3,4,5}


SETS
X text,
Y text,
myset set<int>,
PRIMARY KEY (X,Y)
);

b:myset:1

b:myset:2

X | Y | myset
---+---+-----------a | b | {1,2}
a | c | {3,4,5}

c:myset:3

c:myset:4

c:myset:5

a


LISTS
X text,
Y text,
mylist list<int>,
PRIMARY KEY (X,Y)
);


LISTS
X text,
Y text,
mylist list<int>,
PRIMARY KEY (X,Y)
);

X | Y | mylist
---+---+-----------a | b | [1,2]


LISTS
X text,
Y text,
mylist list<int>,
PRIMARY KEY (X,Y)
);

X | Y | mylist
---+---+-----------a | b | [1,2]

b:mylist:f7e5450039..8d
a

b:mylist:f7e5450139..8d

1

2


MAPS
X text,
Y text,
mymap map<text,int>,
PRIMARY KEY (X,Y)
);


MAPS
X text,
Y text,
PRIMARY KEY (X,Y)
);

X | Y | mymap
---+---+-----------a | b | {m:1,n:2}
a | c |{n:3,p:4,q:5}


MAPS
X | Y | mymap
---+---+-----------a | b | {m:1,n:2}
a | c |{n:3,p:4,q:5}

X text,
Y text,
PRIMARY KEY (X,Y)
);

b:mymap:m
a

b:mymap:n

c:mymap:n

c:mymap:p

c:mymap:q

1

2

3

4

5


Peek Behind the Scenes! Do it!
(in cqlsh)
CREATE KEYSPACE test WITH replication =
{'class': 'SimpleStrategy', 'replication_factor': 1};
USE test;
CREATE TABLE stuff ( a int, b int, myset set<int>,
mylist list<int>, mymap map<int,int>, PRIMARY KEY (a,b));
UPDATE stuff SET myset = {1,2}, mylist = [3,4,5], mymap = {6:7,8:9} WHERE a = 0
AND b = 1;
SELECT * FROM stuff;
(in cassandra-cli)
use test;
list stuff ;
(in cqlsh)
SELECT key_aliases,column_aliases from system.schema_columnfamilies WHERE
keyspace_name = 'test' AND columnfamily_name = 'stuff';

…you already know
• CQL is a reintroduction of schema
• CQL creates a common data modeling language
• CQL is a best-practices Cassandra interface

.

…you already know
…now you know
• CQL let’s you take advantage of the C* Data structure

.

…you already know
…now you know
• CQL let’s you take advantage of the C* Data structure
…but also
• CQL protocol is binary and therefore interoperable with
any language
• CQL is asynchronous and fast (Thrift transport layer is
synchronous)
• CQL allows the possibility for prepared statements

Thanks!
Follow me on Twitter @JnBrymn
Check out the OpenSource Connection Blog
http://www.opensourceconnections.com/blog/


Understanding How CQL3 Maps to Cassandra's Internal Data Structure

In this document