Highly scalable caching service on cloud - Redis

HUAWEI TECHNOLOGIES CO., LTD.
www.huawei.com
Highly Scalable Caching Service on Cloud
Krishna Kumar & Irfan Rehman
Huawei India Cloud Platforms Team

HUAWEI TECHNOLOGIES CO., LTD. 2
We will discuss
 Introduction
 In-memory caching
 Industry use cases
 Types of cloud service provisioning
 The problem to scale..
 Resourse scheduler - Solution 1
 Resourse scheduler - Solution 2
 Demo
 Q&A

Introduction to Cloud Caching Service
Cache 
Component that stores data so future requests for that data can be served faster.
Distributed Cache 
Extension of the traditional concept of cache used in a single locale.
A distributed cache may span multiple servers so that it can grow in size and in
transactional capacity. It is mainly used to store application data residing
in database and web session data. The distributed cache architecture is built on
scale-out, with inherent partitioning capabilities to spread the workload.
Cloud Service 
Any resource that is provided over the Internet. The most common cloud service
resources are Software as a Service (SaaS), Platform as a Service (PaaS) and
Infrastructure as a Service (IaaS).
users
cache

• We focus on in-memory caches and exclude
others (ssd, multi-tier, etc) from our discussion

Major in-Memory Cache Uses Cases
 Database offloading
in case of slow database or disk access
 Web content caching
in case of high latency and narrow bandwidth to actual servers
 Low latency data access
for time critical operations
 Pub/Sub mechanisms
for multiple services and multiple consumers
 Message queues
distributed network message passing mechanisms
 CDN caches
in-memory caches especially for content index and ranking, not
necessarily for content itself
 Caching and computation
sorted sets, lists, queues, etc

Leading Industry Cache Offerings
 Redis
 Memcached
 Apache Hbase
 CounchBase
 Basho Riak
 Oracle Coherence
 DataStax
 Cloudant
 AeroSpike
See other players in the domain of caching:
https://www.g2crowd.com/products/redis/competitors/alternatives

• We chose redis for cloud integration evaluation..

 Most popular in-memory cache in the world
 Redis is an open source (BSD licensed), in-memory data structure store, used as database,
cache and message broker. It supports data structures such as strings, hashes, lists, sets,
sorted sets with range queries, bitmaps, hyperloglogs and geospatial indexes with radius
queries. Redis has built-in replication, Lua scripting, LRU eviction, transactions and
different levels of on-disk persistence, and provides high availability via Redis Sentinel and
automatic partitioning with Redis Cluster.
Web
App
DB
Web
App
Web
App
Web
App
Volatile/temp data
Why Redis NoSQL?
 No lock execution
 Effectively uses individual core per
instance
 Lot of data structure use cases
 Blazingly fast for a single core
performer

• The problem to scale…

Redis Service in cloud – major Vendors
1) Amazon - Elastic Cache: it easy to deploy, operate, and scale an in-memory cache in the cloud. It
support Redis Master/Slavce replication and Multi AZ - https://aws.amazon.com/elasticache/
2) Microsoft - gives you Redis reliable hosting and monitoring - https://azure.microsoft.com/en-
us/documentation/services/redis-cache/
3) Google – Deploy Redis on compute Engine. With Redis-as-a-Service, Redislabs will manage and
monitor your instance on Google Cloud Platform. https://cloud.google.com/solutions/redis/
4) IBM - Support Redis service on SoftLayer -
https://marketplace.ibmcloud.com/apps/87#!overview
5) RedHat OpenShift – Redis support for community-contributed cartridges -
https://blog.openshift.com/enabling-redis-for-your-app/
6) Digital Ocean – Supports using Redis in the cloud -
https://www.digitalocean.com/community/tutorials/how-to-install-and-use-redis
7) Alibaba Aliyun – Aliyun Key-Value Store (KVStore for short) is an online Key-Value storage service
compatible with the open-source Redis protocol -
http://www.aliyun.com/product/kvstore/?lang=en
8) Redis Labs – Enterprise cluster; offered as a fully managed service, also used as third party
offering by other vendors.

One view of Redis service provisioning in cloud
 Instance on Bare metal
- Toughest to maintain, but provides best performance
- A single instance wastes rest of the cpu resources (complicated
multi-tenancy)
 Instance on Virtual Machine
- Lesser performance
- Scaling means provisioning another VM
 Instance in a container
- Lesser performance, given the network translations
- Scaling is easy, but seamless is difficult if the redis cluster is non
uniform (server is different type, proxy is another, etc)
• Eventually all the above is resources.
• Service Instances could be placed anywhere in the pool.
• In fact different types could coexist, and communicate, if the
network allows so.

• The problem to scale given resources..
needs resource scheduling…

Redis service instance topologies
When a customer requests for a redis instance from cloud it could be one for the following configuration.
Single Redis-Server
•CPU = 1
•mem = variable
•Disk = optional
Master/Slave Redis
• Primarily for replication
• CPU= 1 + N Slaves
• mem = variable
• Disk = optional
master
Slave 1 Slave 2
Proxy
Redis Cluster with
Proxy
• Only Using twem proxy or
Codis for data shard
• Popular with older version
of redis eg:2.8
• CPU= Number of nodes
Redis 3.0 Cluster
•Latest offering from Redis ; No proxy required.
•Possible to horizontally scale ; CPU= Number of nodes

 Proxy based sharding
 Twitter handled redis cluster by implementing twemproxy
 Shard aware clients
 External Cluster manager
 Wayfair an online furniture store managed redis cluster using zookeeper
 Enterprise Cluster
 Redis Labs Enterprise cluster - https://redislabs.com/redis-cloud
 Redis 3.0 http://redis.io/download
How traditionally Industry handled Redis so far…

• We tried couple of solutions using
mesos and kubernetes…

Mesos Kubernetes
 Apache Mesos is a cluster manager that provides efficient
resource isolation and sharing across distributed
applications, or frameworks.
 Apache Mesos abstracts CPU, memory, storage, and other
compute resources away from machines (physical or
virtual), enabling fault-tolerant and elastic distributed
systems.
 Google Kubernetes brings 'Google style' cluster
management capabilities to the world of Docker
containers on the data center scenarios.
 For production Google uses Omega/Borg, not Kubernetes
which is open sourced in June 2014.
• Mesos using same principle as Linux Kernel but higher
layer of abstraction.
• It can run Hadoop, MPI, Hypertable, Spark, and other
frameworks on a dynamically shared pool of nodes.
• Manage a cluster of Linux containers as a single system.
• There is a Kubernetes Framework for Mesos by
Mesosphere.
 Was developed at the University of California, Berkeley
year 2009 developed by Benjamin Hindman, Andy
Konwinski, Matel Zaharia as well as professor Ion Stoica
 Become top level Apache project from July 2013
 Project Started at Google
 Recently launched 1.0 production code (2nd half 2015)
 Getting more popular day by day along with Docker.
Works with various Frameworks like Marathon, Cronos. You can
write your own. There is a core algorithm(DRF) to share
resources and you can define your own scheduling policy.
Frameworks does Fault Tolerance, Load balancing, Quotas,
Discovery, etc.
Kubernetes establishes robust declarative primitives for
maintaining the desired state requested by the user. Self-
healing mechanisms, such as auto-restarting, re-scheduling,
and replicating containers require active controllers, not just
imperative orchestration.
Introduction

Mesos Kubernetes
The Architecture

Solution 1 : Kubernetes (Cluster manager)
 We could provide redis instance in docker containers
 Kubernetes has a Mesos framework which can co-exist with
other frameworks/workloads in the same Mesos cluster
 The concept of ‘POD’ can be used to club multiple containers
together. Kubernetes ensures that these containers are started
on the same slave
 Kubernetes automatically provides fault- tolerance
Why kubernetes?

Mesos
Master
Kubernetes
Framework for
Mesos
Cloud service
broker Kube- Executor kube- Executor Kube- Executor Kube- Executor
Redis
Request
from cloud
Each Service instance will be a POD, provisioned
for given type
Solution 1 : Kubernetes as fw of mesos

HUAWEI TECHNOLOGIES CO., LTD. 20Page 20
 Cannot add additional containers in a already running POD. Very
necessary for Redis 3.0 horizontal scalability. Below discussion in
google groups explains the implementation difficulties in
Kubernetes https://groups.google.com/forum/#!topic/google-
containers/_Gz9dH2hqg4
 However it’s a popular demand from the kubernetes community
 Limited Flexibility by using Kubernetes Framework
Solution 1 : Kubernetes (Cluster manager) Contd..
Limitations..

 Implement a simplistic Mesos framework for Long Running
Process
 Use “mesos-go” an opensource framework library written in
golang for development
 Check feasibility for vertical scaling of redis-instances
 Collect statistics against each redis instance
 Enable the fw to ensure instances can communicate to create a
cluster (user 3.0)
Solution 2: Mesos Framework dealing directly with Redis
Instances
Scope

New Redis Mesos Framework
Mesos
Master
Redis
Framework for
Mesos
Cloud service
broker Redis Executor Redis Executor Redis Executor Redis Executor
Redis
Request
from cloud

Redis Executor
Redis Executor
POC : Design of Simple Redis Framework
Redis Scheduler
http server
Monitor
Master
Select offer
&
Submit tasks
Collect Stats
Redis Executor
Monitor
pid=123
http server
UI
PID = 123
PID = 124
PID = 125
Monitor
pid=124
Monitor
pid=125
Launch Task
Mesos
Master
Mesos Slaves

 To build a Mesos framework we have to develop two components.
 Scheduler – Talk to the mesos master and submit tasks
 Executor – Execute your tasks at the slave
 By Default Mesos considers each work load is a task not LRP (Long Running Process)
What is a Mesos framework (a developers perspective):
Usual Logic of Scheduler Usual Logic of Executor
Loop (Wait for offer from Mesos Master)
if (offer.Resource > Task.Resource) then
Select Offer
else
Reject Offer
Repeat
Loop (wait for tasks to Launch )
Update task.State = STARTED
execute task
Update task.State = Finished / Error
Repeat
Instances

 Its easier to write a Mesos framework
 We have better controller & flexibility over the running redis
instances
 We can collect better statistics on the running tasks
 We can plan migration of running tasks among different slaves
 We can group bunch of running processes and allocate them in a
single slave (like kubernetes POD)
 We would be able to offer Redis 3.0 and scale Horizontally
 We could use any container technology of our choice like Docker
or RKT or RunC in bare-metal servers for additional instance
security
WHY and what more can we do?
Instances

DEMO

Q&A

Astro Project – Spark SQL on Hbase
Astro is fully distributed SQL on Hbase solution based on Spark Ecosystem
100% Open source
Any contributions are welcome.
HBase
Spark
Astro
Spark Application
SQL ML Graph Stream
Community
Package
•Astro query processing is fully distributed.
•CRUD(Create, Read, Update and Delete)
operations support
•Fully utilizing Spark SQL query optimizer.
•Advanced optimizations to push down
computations close to storage.
•Co-Location along with Hbase.
As Astro plugs into spark, Astro query
results can be seamlessly used for
further in-depth analysis using Spark SQL
DataFrame, MLLib, GraphX etc…
Address：
https://github.com/HuaweiBigData/astro
Compared to existing SQL on Hbase solution the major Advantages are:

Highly scalable caching service on cloud - Redis

More Related Content

What's hot

Viewers also liked

Similar to Highly scalable caching service on cloud - Redis

More from Krishna-Kumar

Recently uploaded

Highly scalable caching service on cloud - Redis