Software Defined Storage - Open Framework and Intel® Architecture Technologies

Software Defined Storage - Open Framework
and Intel® Architecture Technologies
Anjaneya “Reddy” Chagam
Principal Engineer, Intel Data Center Group
Shayne Huddleston
Infrastructure Architect, Information Services, Oregon State University
DATS001

2
Agenda
•The problem
•Software Defined Storage (SDS) vision
•Why SDS matters?
•Customer perspective
•SDS gaps and Intel response
•Intel role in SDS
•Summary

3
Agenda
•The problem
•Why SDS matters?
•Summary

4
The Problem:
From 2013 to 2020, the digital universe will grow by a factor of 10, from 4.4 ZB to 44 ZB
It more than doubles every two years.
Data needs are growing at a rate unsustainable with today’s infrastructure and labor costs
Source: IDC –The Digital Universe of Opportunities: Rich Data and the Increasing Value of the Internet of Things -April 2014
…cost challenges continue to grow…
….data complexity is increasing
…. and storage accounts for 40% of the data center budget.

5
The Problem with Storage Infrastructure
Storage Silos (Traditional)
•Application mapped to specific appliance
•Storage resources optimized to run specific workload
•Isolated storage resources
Challenges
•Cost of managing diverse storage solutions
-Data Growth
-Maintenance, Operations & Support
-Infrastructure
•Vendor lock-in
-Limited scalability
-Flexibility to innovate
•Need for massively shared data
-But not yet cloud ready
Traditional storage management is too complex and inefficient

6
Agenda
•The problem
•Why SDS matters?
•Summary

7
Software Defined Infrastructure (SDI)
PROVISIONING MANAGEMENT
Orchestration provisions, manages and optimally allocates resources based on the unique requirements of an application
POOLED RESOURCES
Network, Storage and Compute elements are abstracted into resource pools
Storage
Network
Compute
Services Delivery
Resource Pool
Orchestration Software
Infrastructure Attributes
Application A
Application B
Application C
Application D
Power Performance Security Thermals Utilization Location
SERVICE ASSURANCE
Policies and intelligent monitoring triggerdynamic provisioning and service assurance as applications are automatically deployed and maintained

8
SDS – A Key Component of SDI
Dynamic, policy-driven storage resource management
Services Delivery
Software Defined Storage Controller
Orchestration Software
Infrastructure Attributes
App #1 App #2 App #3 App #4
Power Performance Thermals Utilization Location
Storage Systems
Latency Durability
SDS is a framework that delivers a scalable, cost-effective solution to serve
the needs of tomorrow’s Data Center
Abstracting Software from Hardware,
providing flexibility & scalability
Aggregating diverse provider solutions,
increasing flexibility and drive down costs
Provisioning resources dynamically (pay-as-you-
grow) increasing efficiency
Orchestrating application access to diverse
storage systems through Service Level
Agreements (SLAs), increasing flexibility and
handle data complexity

9
SDS Architecture
Applications
SDS Controller
Node
Node
Node
Orchestrator
Storage System
[Capacity]
Storage System
[Performance]
Data Services
Storage System
[SAN]
Storage System
[NAS]
SDS Controller
•Visibility and Control of ALL storage resources
•Communication between Apps, Orchestrator and Storage Systems
•Allocates storage resources to meet SLA’s
Node
Node
JBOD
Data Services
•Application that runs in data plane to optimize storage
•Ex: Predictive Analytics
•Ex: De-Duplication
•Ex: Tiering
………..
………..
SDS : Consolidated Management of Scale-Out and Scale-Up Storage and plug into SDI
Compute Controller
SDN Controller
NAS
SAN
Northbound API
Southbound API

10
Agenda
•The problem
•Why SDS matters?
•Summary

11
Why SDS Matters?
Storage System
[Distributed]
Storage System 1
[SAN]
Storage System
[NAS]
Storage System
[Distributed]
Storage System
[Distributed]
SDS Controller
High Perf
IOPS –10K
(DC1)
High Perf
IOPS –5K
(DC2)
Med Perf
Tpt –100Mbps
(DC2)
Med Perf
Tpt –50 Mbps
(DC1)
Capacity
Tpt –5 Mbps
(DC1)
$$$ Gold
$$ Silver
$ Bronze
Example: Application provides storage pool
1
2
Gold
App 1
(OLTP)
App 2 (Backup)
App 3
(Sharing)
Bronze
Silver
Discover
Compose
3
Assign

12
Why SDS Matters?
Storage System
[Distributed]
Storage System 1
[SAN]
Storage System
[NAS]
Storage System
[Distributed]
Storage System
[Distributed]
SDS Controller
High Perf
IOPS –10K
(DC1)
High Perf
IOPS –5K
(DC2)
Med Perf
Tpt –100Mbps
(DC2)
Med Perf
Tpt –50 Mbps
(DC1)
Capacity
Tpt –5 Mbps
(DC1)
$$$ Gold
$$ Silver
$ Bronze
1
2
App 1
(OLTP)
App 2 (Backup)
App 3
(Sharing)
Bronze
Silver
Discover
Compose
3
Assign

13
Why SDS Matters?
Storage System
[Distributed]
Storage System 1
[SAN]
Storage System
[NAS]
Storage System
[Distributed]
Storage System
[Distributed]
SDS Controller
High Perf
IOPS –10K
(DC1)
High Perf
IOPS –5K
(DC2)
Med Perf
Tpt –100Mbps
(DC2)
Med Perf
Tpt –50 Mbps
(DC1)
Capacity
Tpt –5 Mbps
(DC1)
$$$ Gold
$$ Silver
$ Bronze
1
2
App 1
(OLTP)
App 2 (Backup)
App 3
(Sharing)
Silver
Discover
Compose
3
Assign

14
Why SDS Matters?
Storage System
[Distributed]
Storage System 1
[SAN]
Storage System
[NAS]
Storage System
[Distributed]
Storage System
[Distributed]
SDS Controller
High Perf
IOPS –10K
(DC1)
High Perf
IOPS –5K
(DC2)
Med Perf
Tpt –100Mbps
(DC2)
Med Perf
Tpt –50 Mbps
(DC1)
Capacity
Tpt –5 Mbps
(DC1)
$$$ Gold
$$ Silver
$ Bronze
1
2
App 1
(OLTP)
App 2 (Backup)
App 3
(Sharing)
Discover
Compose
3
Assign

15
Why SDS Matters?
Storage System
[Distributed]
Storage System 1
[SAN]
Storage System
[NAS]
Storage System
[Distributed]
Storage System
[Distributed]
SDS Controller
High Perf
IOPS –10K
(DC1)
High Perf
IOPS –5K
(DC2)
Med Perf
Tpt –100Mbps
(DC2)
Med Perf
Tpt –50 Mbps
(DC1)
Capacity
Tpt –5 Mbps
(DC1)
$$$ Gold
$$ Silver
$ Bronze
Example: Application provides SLO attributes (performance)
1
2
IOPS=7000
App 1
App 2
App 3
Tpt -2 Mbps
Tpt –30Mbps
Discover
Compose
3
Assign

16
Why SDS Matters?
Storage System
[Distributed]
Storage System 1
[SAN]
Storage System
[NAS]
Storage System
[Distributed]
Storage System
[Distributed]
SDS Controller
High Perf
IOPS –10K
(DC1)
High Perf
IOPS –5K
(DC2)
Med Perf
Tpt –100Mbps
(DC2)
Med Perf
Tpt –50 Mbps
(DC1)
Capacity
Tpt –5 Mbps
(DC1)
$$$ Gold
$$ Silver
$ Bronze
1
2
App 1
App 2
App 3
Tpt -2 Mbps
Tpt –30Mbps
Discover
Compose
3
Assign

17
Why SDS Matters?
Storage System
[Distributed]
Storage System 1
[SAN]
Storage System
[NAS]
Storage System
[Distributed]
Storage System
[Distributed]
SDS Controller
High Perf
IOPS –10K
(DC1)
High Perf
IOPS –5K
(DC2)
Med Perf
Tpt –100Mbps
(DC2)
Med Perf
Tpt –50 Mbps
(DC1)
Capacity
Tpt –5 Mbps
(DC1)
$$$ Gold
$$ Silver
$ Bronze
1
2
App 1
App 2
App 3
Tpt –30Mbps
Discover
Compose
3
Assign

18
Why SDS Matters?
Storage System
[Distributed]
Storage System 1
[SAN]
Storage System
[NAS]
Storage System
[Distributed]
Storage System
[Distributed]
SDS Controller
High Perf
IOPS –10K
(DC1)
High Perf
IOPS –5K
(DC2)
Med Perf
Tpt –100Mbps
(DC2)
Med Perf
Tpt –50 Mbps
(DC1)
Capacity
Tpt –5 Mbps
(DC1)
$$$ Gold
$$ Silver
$ Bronze
1
2
App 1
App 2
App 3
Discover
Compose
3
Assign

19
Why SDS Matters?
Storage System
[Distributed]
Storage System 1
[SAN]
Storage System
[NAS]
Storage System
[Distributed]
Storage System
[Distributed]
SDS Controller
High Perf
IOPS –10K
(DC1)
High Perf
IOPS –5K
(DC2)
Med Perf
Tpt –100Mbps
(DC2)
Med Perf
Tpt –50 Mbps
(DC1)
Capacity
Tpt –5 Mbps
(DC1)
$$$ Gold
$$ Silver
$ Bronze
Example: Application provides SLO attributes (performance, client caching)
1
2
IOPS=7000
App 1
Discover
Compose
3
Assign
Client Cache = 200GBApp1

20
Why SDS Matters?
Storage System
[Distributed]
Storage System 1
[SAN]
Storage System
[NAS]
Storage System
[Distributed]
Storage System
[Distributed]
SDS Controller
High Perf
IOPS –10K
(DC1)
High Perf
IOPS –5K
(DC2)
Med Perf
Tpt –100Mbps
(DC2)
Med Perf
Tpt –50 Mbps
(DC1)
Capacity
Tpt –5 Mbps
(DC1)
$$$ Gold
$$ Silver
$ Bronze
Example: Application provides SLO attributes (performance, client caching)
1
2
App 1
Discover
Compose
3
Assign
Client Cache = 200GBApp1

21
Why SDS Matters?
Storage System
[Distributed]
Storage System 1
[SAN]
Storage System
[NAS]
Storage System
[Distributed]
Storage System
[Distributed]
SDS Controller
High Perf
IOPS –10K
(DC1)
High Perf
IOPS –5K
(DC2)
Med Perf
Tpt –100Mbps
(DC2)
Med Perf
Tpt –50 Mbps
(DC1)
Capacity
Tpt –5 Mbps
(DC1)
$$$ Gold
$$ Silver
$ Bronze
1
2
IOPS=170K (R/W)
App 1
Discover
Compose
3
Assign
Example: Application provides SLO attributes (ephemeral)
Ephemeral

22
Why SDS Matters?
Storage System
[Distributed]
Storage System 1
[SAN]
Storage System
[NAS]
Storage System
[Distributed]
Storage System
[Distributed]
SDS Controller
High Perf
IOPS –10K
(DC1)
High Perf
IOPS –5K
(DC2)
Med Perf
Tpt –100Mbps
(DC2)
Med Perf
Tpt –50 Mbps
(DC1)
Capacity
Tpt –5 Mbps
(DC1)
$$$ Gold
$$ Silver
$ Bronze
1
2
IOPS=170K (R/W)
Discover
Compose
3
AssignApp1
Example: Application provides SLO attributes (ephemeral)

23
Why SDS Matters?
Storage System
[Distributed]
Storage System 1
[SAN]
Storage System
[NAS]
Storage System
[Distributed]
Storage System
[Distributed]
SDS Controller
High Perf
IOPS –10K
(DC1)
High Perf
IOPS –5K
(DC2)
Med Perf
Tpt –100Mbps
(DC2)
Med Perf
Tpt –50 Mbps
(DC1)
Capacity
Tpt –5 Mbps
(DC1)
$$$ Gold
$$ Silver
$ Bronze
1
2
IOPS=170K (R/W)
App 1
Discover
Compose
3
Assign
Example: Application provides SLO attributes (ephemeral, local protection)
Persistent, Local Protection

24
Why SDS Matters?
Storage System
[Distributed]
Storage System 1
[SAN]
Storage System
[NAS]
Storage System
[Distributed]
Storage System
[Distributed]
SDS Controller
High Perf
IOPS –10K
(DC1)
High Perf
IOPS –5K
(DC2)
Med Perf
Tpt –100Mbps
(DC2)
Med Perf
Tpt –50 Mbps
(DC1)
Capacity
Tpt –5 Mbps
(DC1)
$$$ Gold
$$ Silver
$ Bronze
1
2
IOPS=170K (R/W)
Discover
Compose
3
AssignApp1
Example: Application provides SLO attributes (ephemeral, local protection)

25
Why SDS Matters?
Storage System
[Distributed]
Storage System 1
[SAN]
Storage System
[NAS]
Storage System
[Distributed]
Storage System
[Distributed]
SDS Controller
High Perf
IOPS –10K
(DC1)
High Perf
IOPS –5K
(DC2)
Med Perf
Tpt –100Mbps
(DC2)
Med Perf
Tpt –50 Mbps
(DC1)
Capacity
Tpt –5 Mbps
(DC1)
$$$ Gold
$$ Silver
$ Bronze
1
2
IOPS=9000
App 1
Discover
Compose
3
Assign
Example: Application provides SLO attributes (IOPS, noisy neighbor policy)
Compute Controller

26
Why SDS Matters?
Storage System
[Distributed]
Storage System 1
[SAN]
Storage System
[NAS]
Storage System
[Distributed]
Storage System
[Distributed]
SDS Controller
High Perf
IOPS –10K
(DC1)
High Perf
IOPS –5K
(DC2)
Med Perf
Tpt –100Mbps
(DC2)
Med Perf
Tpt –50 Mbps
(DC1)
Capacity
Tpt –5 Mbps
(DC1)
$$$ Gold
$$ Silver
$ Bronze
1
2
IOPS=9000
App 1
Discover
Compose
3
Assign
App1
Example: Application provides SLO attributes (IOPS, noisy neighbor policy)
Compute Controller
Qemu QoS
Max IOPS=9000

27
Why SDS Matters?
Storage System
[Distributed]
Storage System 1
[SAN]
Storage System
[NAS]
Storage System
[Distributed]
Storage System
[Distributed]
SDS Controller
High Perf
IOPS –10K
(DC1)
High Perf
IOPS –5K
(DC2)
Med Perf
Tpt –100Mbps
(DC2)
Med Perf
Tpt –50 Mbps
(DC1)
Capacity
Tpt –5 Mbps
(DC1)
$$$ Gold
$$ Silver
$ Bronze
1
2
Tpt –2 Mbps
App 1
Discover
Compose
3
Assign
Example: Application provides SLO attributes (Throughput, noisy neighbor policy)
SDN

28
Why SDS Matters?
Storage System
[Distributed]
Storage System 1
[SAN]
Storage System
[NAS]
Storage System
[Distributed]
Storage System
[Distributed]
SDS Controller
High Perf
IOPS –10K
(DC1)
High Perf
IOPS –5K
(DC2)
Med Perf
Tpt –100Mbps
(DC2)
Med Perf
Tpt –50 Mbps
(DC1)
Capacity
Tpt –5 Mbps
(DC1)
$$$ Gold
$$ Silver
$ Bronze
1
2
Tpt –2 Mbps
App 1
Discover
Compose
3
AssignApp1
Example: Application provides SLO attributes (Throughput, noisy neighbor policy)
SDN
tc ul rate 2048kbit

29
Why SDS Matters?
Storage System
[Distributed]
Storage System 1
[SAN]
Storage System
[NAS]
Storage System
[Distributed]
Storage System
[Distributed]
SDS Controller
High Perf
IOPS –10K
(DC1)
High Perf
IOPS –5K
(DC2)
Med Perf
Tpt –100Mbps
(DC2)
Med Perf
Tpt –50 Mbps
(DC1)
Capacity
Tpt –5 Mbps
(DC1)
$$$ Gold
$$ Silver
$ Bronze
1
2
Tpt –2 Mbps
App 1
Discover
Compose
3
Assign
Example: Application provides SLO attributes (Performance but storage is full)
Free Pool
2
Policies
Policy #X
If 80% capacity
Then mark system unavailable
81% utilization

30
Why SDS Matters?
Storage System
[Distributed]
Storage System 1
[SAN]
Storage System
[NAS]
Storage System
[Distributed]
Storage System
[Distributed]
SDS Controller
High Perf
IOPS –10K
(DC1)
High Perf
IOPS –5K
(DC2)
Med Perf
Tpt –100Mbps
(DC2)
Med Perf
Tpt –50 Mbps
(DC1)
Capacity
Tpt –5 Mbps
(DC1)
$$$ Gold
$$ Silver
$ Bronze
1
2
App 1
Discover
Compose
3
Assign
Free Pool
2
Policies
Policy #X
If 80% capacity
81% utilization

31
Why SDS Matters?
Storage System
[Distributed]
Storage System 1
[SAN]
Storage System
[NAS]
Storage System
[Distributed]
Storage System
[Distributed]
SDS Controller
High Perf
IOPS –10K
(DC1)
High Perf
IOPS –5K
(DC2)
Med Perf
Tpt –100Mbps
(DC2)
Med Perf
Tpt –50 Mbps
(DC1)
Capacity
Tpt –5 Mbps
(DC1)
$$$ Gold
$$ Silver
$ Bronze
1
2
App 1
Discover
Compose
3
Assign
Free Pool
2
Policies
Policy #X
If 80% capacity

32
Why SDS Matters?
Storage System
[Distributed]
Storage System 1
[SAN]
Storage System
[NAS]
Storage System
[Distributed]
Storage System
[Distributed]
SDS Controller
High Perf
IOPS –10K
(DC1)
High Perf
IOPS –5K
(DC2)
Med Perf
Tpt –100Mbps
(DC2)
Med Perf
Tpt –50 Mbps
(DC1)
Capacity
Tpt –5 Mbps
(DC1)
$$$ Gold
$$ Silver
$ Bronze
1
2
Discover
Compose
3
Assign
Free Pool
2
Policies
Policy #X
If 80% capacity
App 1

33
Agenda
•The problem
•Why SDS matters?
•Summary

34
Perspectives from Oregon State University (OSU) IT on SDS
20

35
Oregon State University -Introduction
A top research university
•28,000 Students, 3,500 faculty, 5 datacenters
•TopCarnegieFoundation class of researchuniversities
•One of only two universities in the U.S. to be a Land, Sea, Space and Sun Grant university
•Statewide economic footprint of $2.06 Billion
•15 Agricultural Experiment Stations; two remote campuses, Hatfield Marine Sciences Center in Newport,, and OSU-Cascades in Bend
21

36
Oregon State University -Information Services
•Central IT provider for OSU (Infrastructure as a Service -storage, compute, networking, facilities)
•Mission is to enable and support Students, Faculty & Science
•Multitude of different workloads that we must support
-Long term digital archive
durable, replicated, WORM
-Virtual Machines
-Virtual Desktops (VDI)
-Research workloads
Genome sequencers generate 20TB/hour sustained high bandwidth write
-OLTP -Oracle, MSSQL, mySQL
-Business analytics (Hadoop, etc.)
22

37
Oregon State University –DC Infrastructure
•3 central data centers on campus
•Infrastructure
-Heavily virtualized
-High availability metro cluster architecture
-Home grown orchestration layer
-Puppet configuration management
-Active/active replicated block storage
-SAN, NAS storage islands
-Distributedstorage (piloting)
23

38
Oregon State University –IT Challenges
24
•Storagedemand is growing faster
-Growing at over 40%/year
•Storage is not optimized for diverse workloads
•Traditional storage management is costlyand hard to scale
•Lot of touch points to administer, provision, and consume storage resources
•Allocation of storage resources to user workloads is 1 to 1
•IT budgetis relatively flat
•Majority of storage cost is hardware and operations

39
Oregon State University –Focus Areas & Plans
25
•Storage growth explosion
-Shift to cost optimized, “pay as you grow” storage architecture
-Distributed storage on standard high volume servers (open source focus)
•Storage optimized for diverse workloads
-SLA and policy driven storage resource allocation
-Storage allocation based on workloads, performance, cost and resiliency requirements
•Storage management
-All storage operations managed by single controller in the DC
Administrator should not have to go beyond the controller except in extraordinary circumstances
-Recognition and classification of performance problems in central dashboard
-Fully automated provisioning and simplified maintenance
New capacity automatically added to appropriate service and controlled by policies
OSU is collaborating with Intel to address storage challenges via open source SDScontroller initiative

40
Agenda
•The problem
•Why SDS matters?
•Summary

41
SDS Gaps
SDS Controller
Clear Standards
Standards exist, but they are not focused on SDS, not organized together, and are not complete
Industry wide focus needed to develop standards by either enhancing or developing new standards (e.g., SNIA, TOSCA, DMTF)
Node
Applications
Northbound API
Southbound API
Node
Node
SDS Controller
SDS Controller
StorageSystem
(Capacity)
Storage System
(Performance)
Node
Arrays
Orchestrator

42
Storage Control Plane Standards –State & Gaps
Intel plans to work with standard bodies to address SDS gaps
Node
Applications
Northbound API
Southbound API
Node
Node
SDS Controller
SDS Controller
Storage System
(Capacity)
Storage System
(Performance)
Node
Arrays
Orchestrator
Standard
Gap
OASIS TOSCA
Applications provide storage requirements using SLOs
IETF
Network focus-storage requirements (e.g. policies) not comprehended
CDMI
Object storage focus –need extensions for block, file, etc.
SMI-S
Appliance focus -requires changes for distributed storage usage

43
SDS Gaps
Clear Standards
SDS Controller
Open, federated “control plane” with pluggable architecture is needed for ecosystem innovation
Node
Applications
Northbound API
Southbound API
Node
Node
SDS Controller
SDS Controller
StorageSystem
(Capacity)
Storage System
(Performance)
Node
Arrays
Orchestrator
Open SDS Controller does not exist

44
SDS Controller
Storage
Orchestration
SDN Controller
Storage Systems
Distributed Storage
SLOs
File/Block
Object
Service/ Tenant Portal
REST APIs
Users
Storage Hardware, Media
Unprovisioned
Gold
Latency –5ms
IOPS –20000
Access -block
Silver
Latency –10ms
IOPS –1000
Access -block
Bronze
Latency –50ms
Throughput –100Mbps
Access -object
Storage Arrays
File/Block
MongoDB*
eCommerce
Media Store
SLOs
SLOs
Distributed Storage
Compute Controller

45
SDS Controller
Storage
Orchestration
SDN Controller
Storage Systems
Distributed Storage
SLOs
File/Block
Object
SLAs
Applica- tions
REST APIs
Users
Unprovisioned
Gold
Latency –5ms
IOPS –20000
Access -block
Silver
Latency –10ms
IOPS –1000
Access -block
Bronze
Latency –50ms
Access -object
Storage Arrays
File/Block
MongoDB*
eCommerce
Media Store
SLOs
SLOs
Distributed Storage
Compute Controller

46
SDS Controller
Storage
Orchestration
SDN Controller
Storage Systems
Distributed Storage
SLOs
File/Block
Object
SLAs
Applica- tions
SLA  SLO
REST APIs
Users
Unprovisioned
Gold
Latency –5ms
IOPS –20000
Access -block
Silver
Latency –10ms
IOPS –1000
Access -block
Bronze
Latency –50ms
Access -object
Storage Arrays
File/Block
MongoDB*
eCommerce
Media Store
SLOs
SLOs
Distributed Storage
Compute Controller

47
SDS Controller
Storage
Orchestration
SDN Controller
Storage Systems
Distributed Storage
SLOs
File/Block
Object
SLAs
Applica- tions
SLA  SLO
REST APIs
Users
Unprovisioned
Gold
Latency –5ms
IOPS –20000
Access -block
Silver
Latency –10ms
IOPS –1000
Access -block
Bronze
Latency –50ms
Access -object
Storage Arrays
File/Block
MongoDB*
eCommerce
Media Store
Provisioning
QoS
SDS Controller
Control Plane
Policy Mgmt
SLA  Storage SLO
Monitoring
REST APIs
SLOs
SLOs
Distributed Storage
Compute Controller

48
SDS Controller
Storage
Orchestration
SDN Controller
Storage Systems
Distributed Storage
SLOs
File/Block
Object
Data
Plane
SLAs
Applica- tions
SLA  SLO
REST APIs
Users
Unprovisioned
Gold
Latency –5ms
IOPS –20000
Access -block
Silver
Latency –10ms
IOPS –1000
Access -block
Bronze
Latency –50ms
Access -object
Storage Arrays
File/Block
MongoDB*
eCommerce
Media Store
Provisioning
QoS
SDS Controller
Control Plane
Policy Mgmt
SLA  Storage SLO
Monitoring
REST APIs
SLOs
SLOs
Distributed Storage
Compute Controller

49
SDS Controller
Storage
Orchestration
SDN Controller
Storage Systems
Distributed Storage
SLOs
File/Block
Object
Data
Plane
SLAs
Applica- tions
SLA  SLO
REST APIs
Users
Unprovisioned
Data Services (optional)
Gold
Latency –5ms
IOPS –20000
Access -block
Silver
Latency –10ms
IOPS –1000
Access -block
Bronze
Latency –50ms
Access -object
Storage Arrays
File/Block
MongoDB*
eCommerce
Media Store
Provisioning
QoS
SDS Controller
Control Plane
Policy Mgmt
SLA  Storage SLO
Monitoring
REST APIs
SLOs
SLOs
Distributed Storage
Compute Controller

50
Open Source SDS Controller -Prototype
Horizon Dashboard
SDS Controller
Cinder
Manila
Glance
Object Storage
Command Line Interface
Neutron
Nova
Keystone
Ceilometer
Distributed Storage
SAN
Distributed Storage
NAS
Distributed Storage
Distributed Storage
2
1
1
SDS controller discovers storage systems and capabilities (e.g., perf, capacity, tiers, etc.)
Admin composes storage pools (e.g., gold, silver, bronze)
Application requests storage service using SLOs.
Controller allocates storage volume from pool that can best service the request.
Storage gets assigned to Nova or App in VM.
Controller works with compute, network to set QoS.
2
1
1
1
3
3
4
4
4
4
Develop key requirements by working with ecosystem and finalize open source enabling plans in 2H’14

51
Agenda
•The problem
•Why SDS matters?
•Summary

52
How is Intel Helping?
Virtual Storage Manager (VSM)†
Plug-ins
(e.g. Intel ISA-L Open Source Version)
COSBench
Object Storage RA
Virtual Block RA
Silicon innovations, software optimizations to accelerate Open source Software Defined Storage
Swift
Ceph
Cinder
Media Transcoding
VDI
INTEL® TRUE SCALE
INTEL® SSD
INTEL® ETHERNET
Intel® Enterprise Edition for Lustre*software
Intel® Data Plane
Development Kit
•Intel® Intelligent Storage Acceleration Library (Intel® ISA-L)
•Intel® Cache Acceleration Software (Intel® CAS)
STORAGE
Intel® Platform Storage Extensions
Intel Ingredients (Silicon, Platforms, Software)
NETWORK
Intel® Open Network Platform
Intel® QuickAssist Accelerators
Intel® Silicon Photonics
COMPUTE
Intel® Processor Graphics
Intel® Trusted Execution Technology (Intel® TXT)
Intel® AVX
Intel® AES New Instructions
Intel® Media SDK
for Servers
Intel® Advanced Vector Extensions (Intel® AVX);
†Virtual Storage Manager (VSM) to be open sourced in Q4’14
Open Source
(Storage)
Reference Architectures

53
Agenda
•The problem
•Why SDS matters?
•Summary

54
Summary
•Software Defined Storage is needed to address tomorrow’s challenges
-Data needs are growing at a rate unsustainable with today’s infrastructure and labor costs
-Traditional storage silos drives management complexity and inefficient
•SDS is a framework aimed at serving the needs of emerging storage requirements. But there are gaps -
-Industry wide focus is needed to create standards for application and storage system interoperability
-Open, federated “control plane” that provisions, monitors, ensures SLAs are needed
•Intel is working with community and ecosystem to address these gaps

55
Call to Action
•Take advantage of SDS disruptive wave
-Consider SDS POCs in Q1’15 (based on open source controller)
•Engage with the community to address storage challenges
-Provide feedback on storage requirements
-Help us drive open source, standards based Software Defined Storage controller
•Innovate storage offerings with Open source framework

56
Additional Sources of Information
•A PDF of this presentation is available from our Technical Session Catalog: www.intel.com/idfsessionsSF. This URL is also printed on the top of Session Agenda Pages in the Pocket Guide.
•Demos in the showcase –
-Transform storage with Software Defined Storage controller, Swift and Ceph (Booth: 122)
-Fujitsu Hyperscale*:New software-based distributed storage system featuring Intel® Virtual Storage Manager software (Booth: 123)
-Making it Real:Rack Scale Architecture for Software Defined Infrastructure and Orchestration (Booth: 120)

57
Additional Sources of Information
Poster Chats:
•DATC003 -Optimizing Cephwith Intel® Technologies
-Data Center & SDI Community, Station 2, Tuesday, 3:00pm –5:00pm
•DATC010 -Improving Storage Performance Through I/O Hinting
-Data Center & SDI Community, Station 2, Tuesday, 5:00pm –7:00pm

58
Legal Disclaimer
INFORMATION IN THIS DOCUMENT IS PROVIDED IN CONNECTION WITH INTEL PRODUCTS.NO LICENSE, EXPRESS OR IMPLIED, BY ESTOPPEL OR OTHERWISE, TO ANY INTELLECTUAL PROPERTY RIGHTS IS GRANTED BY THIS DOCUMENT.EXCEPT AS PROVIDED IN INTEL'S TERMS AND CONDITIONS OF SALE FOR SUCH PRODUCTS, INTEL ASSUMES NO LIABILITY WHATSOEVER AND INTEL DISCLAIMS ANY EXPRESS OR IMPLIED WARRANTY, RELATING TO SALE AND/OR USEOF INTEL PRODUCTS INCLUDING LIABILITY OR WARRANTIES RELATING TO FITNESS FOR A PARTICULAR PURPOSE, MERCHANTABILITY, OR INFRINGEMENT OF ANYPATENT, COPYRIGHT OR OTHER INTELLECTUAL PROPERTY RIGHT.
A "Mission Critical Application" is any application in which failure of the Intel Product could result, directly or indirectly, in personal injury or death.SHOULD YOU PURCHASE OR USE INTEL'S PRODUCTS FOR ANY SUCH MISSION CRITICAL APPLICATION, YOU SHALL INDEMNIFY AND HOLD INTEL AND ITS SUBSIDIARIES, SUBCONTRACTORS AND AFFILIATES, AND THE DIRECTORS, OFFICERS, AND EMPLOYEES OF EACH, HARMLESS AGAINST ALL CLAIMS COSTS, DAMAGES, AND EXPENSES AND REASONABLE ATTORNEYS' FEES ARISING OUT OF, DIRECTLY OR INDIRECTLY, ANY CLAIM OF PRODUCT LIABILITY, PERSONAL INJURY,OR DEATH ARISING IN ANY WAY OUT OF SUCH MISSION CRITICAL APPLICATION, WHETHER OR NOT INTEL OR ITS SUBCONTRACTOR WAS NEGLIGENT IN THE DESIGN, MANUFACTURE, OR WARNING OF THE INTEL PRODUCT OR ANY OF ITS PARTS.
Intel may make changes to specifications and product descriptions at any time, without notice.Designers must not rely on the absence or characteristics of any features or instructions marked "reserved" or "undefined".Intel reserves these for future definition and shall have no responsibility whatsoever for conflicts or incompatibilities arising from future changes to them.The information here is subject to change without notice.Do not finalize a design with this information.
The products described in this document may contain design defects or errors known as errata which may cause the product to deviate from published specifications.Current characterized errata are available on request.
Contact your local Intel sales office or your distributor to obtain the latest specifications and before placing your productorder.
Copies of documents which have an order number and are referenced in this document, or other Intel literature, may be obtained by calling 1-800-548-4725, or go to:http://www.intel.com/design/literature.htm
Intel, Look Inside and the Intel logo are trademarks of Intel Corporation in the United States and other countries.
*Other names and brands may be claimed as the property of others.
Copyright ©2014 Intel Corporation.

59
Risk Factors
The above statements and any others in this document that refer to plans and expectations for the second quarter, the year and the future are forward- looking statements that involve a number of risks and uncertainties. Words such as “anticipates,” “expects,” “intends,” “plans,”“believes,” “seeks,” “estimates,” “may,” “will,” “should” and their variations identify forward-looking statements. Statements that refer to or are based on projections, uncertain events or assumptions also identify forward-looking statements. Many factors could affect Intel’s actual results, and variances from Intel’s current expectations regarding such factors could cause actual results to differ materially from those expressed in these forward-looking statements. Intel presently considers the following to be important factors that could cause actual results to differ materially from thecompany’s expectations. Demand for Intel's products is highly variable and, in recent years, Intel has experienced declining orders in the traditional PC market segment. Demand could be different from Intel's expectations due to factors including changes in business and economic conditions; consumer confidence or income levels; customer acceptance of Intel’s and competitors’ products; competitive and pricing pressures, including actionstaken by competitors; supply constraints and other disruptions affecting customers; changes in customer order patterns including order cancellations; and changes in the level of inventory at customers. Intel operates in highly competitive industries and its operations have high costs that are either fixed or difficult to reduce in the short term. Intel's gross margin percentage could vary significantly from expectations based on capacity utilization; variations in inventory valuation, including variations related to the timing of qualifying products for sale; changes in revenue levels; segment product mix; the timing and execution of the manufacturing ramp and associated costs; excess or obsolete inventory; changes in unit costs; defects or disruptions in the supply of materials or resources; and product manufacturing quality/yields. Variations in gross margin may also be caused by the timing of Intel product introductions and related expenses, including marketing expenses, and Intel's ability to respond quickly to technological developments and to introduce new products or incorporate new features into existing products, which may result in restructuring and asset impairment charges. Intel's results could be affected by adverse economic, social, political and physical/infrastructure conditions in countries where Intel, its customers or its suppliers operate, including military conflict and other security risks, natural disasters, infrastructure disruptions, health concerns and fluctuations in currency exchange rates. Intel’s results could be affected by the timing of closing of acquisitions, divestitures and other significant transactions. Intel's results could be affected by adverse effects associated with product defects and errata (deviations from published specifications), and by litigation or regulatory matters involving intellectual property, stockholder, consumer, antitrust, disclosure and other issues, such as the litigation and regulatory matters described in Intel's SEC filings. An unfavorable ruling could include monetary damages or an injunction prohibiting Intel from manufacturing or selling one or more products, precluding particular business practices, impacting Intel’s ability to design its products, or requiring other remedies such as compulsory licensing of intellectual property. A detailed discussion of these and other factors that could affect Intel’s results is included in Intel’s SEC filings, including the company’s most recent reports on Form 10-Q, Form 10-K and earnings release.
Rev. 4/15/14

61
SDS –Industry Definitions

62
IDC: “Any storage softwarestack that can be installed onany commodity resources(x86 hardware, hypervisors, or cloud) and/or off-the-shelf computing hardware and used to offer a full suite of storage services and federationbetween the underlying persistent data placement resources to enable data mobility of its tenants between these resources.”

63
SNIA: Software-defined storage (SDS)is a term for computer data storagetechnologies which separate storage hardware fromthe softwarethat manages the storage infrastructure. The software enabling a software-defined storage environment provides policy managementfor feature options such as deduplication, replication, thin provisioning, snapshots and backup.

64
Wikipedia: Software-defined storage (SDS) is a term for computer data storage technologies which separate storage hardware from the software that manages the storage infrastructure. Characteristics: 1) Abstraction of logical storage services and capabilities from the underlying physicalstorage systems, and in some cases poolingacross multiple different implementations. 2) Automationwith policy-drivenstorage provisioning withservice-level agreementsreplacing technology details. 3) Commodity hardware with storagelogic abstracted into a software layer. 4) Scale-outstorage architecture.

65
Wikipedia: Software-defined storage (SDS) is a term for computer data storage technologies which separate storage hardware from the software that manages the storage infrastructure. Characteristics: 1) Abstraction of logical storage services and capabilities from the underlying physicalstorage systems, and in some cases poolingacross multiple different implementations. 2) Automationwith policy-drivenstorage provisioning withservice-level agreementsreplacing technology details. 3) Commodity hardware with storagelogic abstracted into a software layer. 4) Scale-outstorage architecture.
Intel’s definition includes “all” above elements but with emphasis on end to end storage management at a data center level

Software Defined Storage - Open Framework and Intel® Architecture Technologies

Recommended

More Related Content

What's hot (20)

Viewers also liked (20)

Similar to Software Defined Storage - Open Framework and Intel® Architecture Technologies (20)

More from Odinot Stanislas (19)

Recently uploaded (20)

Software Defined Storage - Open Framework and Intel® Architecture Technologies