Flash card architect storage infrastructure in azure

© 2019 Veeam Software. Confidential information. All rights reserved. All trademarks are the property of their respective owners.
Flash Card – Architect
Storage Infrastructure
Prepared by Lai

Structured data
Semi structured
data
Relational data
-with schema
(field/properties)
Ex: SQL database
table with row &
column
Contain tag such as
key/value pairs
Ex: Non relational or
NoSQL data
Example: XML, JSON,
YAML
Unstructured
Data
Files (photos, video)
Office file
Log Files, etc
Types of Data

Storage Solution Scenario 1
Customer require high
read operation –to
query data
Business need high
number of write
operation(changing
inventory)
High throughput & Lo
Latency
Transaction support-
Yes
Answer: Azure Cosmos DB
Semi structured data

Customer require high
read operation with
low latency
Create & update –
infrequent & can
have higher latency
Retrieval by ID: need
to support low latency
& high throughput
No
Answer: Azure Blob Storage with Azure Content Delivery
Network (CDN)
Unstructured data

Read only, complex
analytical queries
across multiple
database
Some latency
expected based on
complex nature of
the queries
Yes
Answer: Azure SQL Database
Structured data

Flash Card – Storage
Account
Prepared by Lai

Scalable
object store
File shares
Messaging store
for reliable
messaging
between app
component
NoSQL store for
schemaless storage
of structured data

Azure Storage Support Blob Type
Hold file up tp 5TB. Files larger than 100MB must be
uploaded as small block, then consolidated into final blob
Hold random access file up to 8 TB. Used primary as
backing storage for VHD
Block Blob
Page Blob
Append Blob
Optimized for append operation. Frequent use for
logging

Independent
Azure resources.
Not in storage
account

Geo-redundant
storage
Locally redundant
storage
Critical Data
Non Critical
Data

Kind of Storage Account
Support all storage types and latest features
(recommended)
Legacy, not support all features
Storage v2
Storage v1
Block blobs and append blobsBlob storage

Flash Card – Azure
Storage Explorer
Prepared by Lai

Azure Storage
Explorer
View all the
storage services
Upload, Edit,
Download,
Copy & delete
Use on :
a) Azure Blob storage-
store unstructured
data
b) Azure table storage –
store noSQL, semi
structured data
c) Azure Files- store files
d) Azure Data Lake
Storage- based on
Apache Hadoop for
store large data
volumes to store
unstructured &
structured data
Included local emulator for dev (no cost):
a) Azure Storage Emulator – emulates Azure Table,
Queue & Blob storage
b) Azurite – based on Node.js. Support Azure Storage
cmd through API

Connection Types to Azure Storage
Explorer
Azure AD- azure tenant
Shared Access Signature
URI- use SAS URI
Storage account name &
Access Key
Access data in Azure
Cosmos DB- use
connection string
Access data in Azure
data Lake -use URI
Attach to local
emulator

Storage Redundancy
Prepared by Lai

Data Replication
Data
Replication
Locally redundant storage
(LRS)
-replicate within same data
center
- Data available if a node
fail
- Data unavailable , if
entire data center fail
Geo redundant storage
(GRS)
-replicate to 2nd region
- Data available when
primary region is failed
Read- Access Geo redundant
storage (RA-GRS)
-read only data in 2nd
location
-data readable but cannot be
modified
Zone redundant storage
(ZRS)
--replicate data sync across
3 storage cluster in single
region.
-resides on own AZ

Locally redundant storage (LRS)
Replicates data &
stores 3 copies across
fault domain (rack of
hw)
Single region Single datacenter
Potential loss of data
when datacenter
outage
3 copies

Zone Redundant Storage (ZRS)
Replicates across 3
storage cluster in a
region
3 copies
Replicate to different
AZ
If outage,data
available from different
AZ
Single region
Not protect from
regional outage

Geographically Redundant Storage (GRS)
Replicates 3x within
primary region,
replicate to secondary
region
6 copies of data
Secondary region is
passive
Two region

Read Access Geo Redundant Storage (RA-GRS)
Replicates data to
secondary region
Suitable for application
(HA)
Control use DNS
failover process
Two region
Read access in
secondary datacenter
in other region
6 copies

Storage Security
Prepared by Lai

Security
Encryption at rest (
SSE with 256 bit AES)
for Azure Storage
VM encrypt using
Azure Disk Encryption
(bitlocker) or dm-
crypt
Encryption in transit
(https)
RBAC and Azure AD
Auditing access using
Storage Analytic
service

Advanced Threat Protection for Azure Storage
Detect anomalies
in account
activities
Available for blob
storage
Security alert are
integrated with
Azure Security
Center
Alert sent by email

Azure Files
Std File share Smb
Direct access via mounted drive in OS or install Azure Files
Sync to synchronize the files between local shares & Azure
Files

Azure File Redundancy

Data Migration Solution

Migration Tool

Command Azure CLI
Create Storage Az storage account create
Create File
Share
Az storage share create
Create VM Az vm create

Secure Access from on-prem
Azure Files use SMB v3
(secure compare to v1)
Use TCP 445
Remove SMB v1:-Remove-
WindowsFeature -Name FS-SMB1
Configure Ip based
firewall rules
Enable secure transfer
settings (request via
https)
Use Azure AD DS
(manage access with role
based access control)
Use share snapshot to
protect against accidental
deletion

Disk Types for VM
Ultra SSD
• Highest performance
• Able to adjust IOPS
throughput while
running
• Only available in
certain region
• Only to ES/DS v3 VM
• Used as data disk
• Don’t support disk
snapshot, VM images,
Scale set, Azure Disk
Encryption, Azure
Backup & ASR
Premium SSD
• High throughput &
IOPS with low latency
• Available on all region
• Can’t adjust
performance without
detach from VM
• Only on larger VM
size
• Guaranteed
throughput
Standard
SSD
Available to any VM or any size
Standard
HDD
• Consistent
Performance
• not
guaranteed
performance
Stored on
conventional
magnetic
disk (less
critical,
dev,test)

Flash Card – Monitor
Azure Storage
Prepared by Lai

Troubleshooting tool
Azure Portal
Azure Storage
Explorer
• Configure metric &
Logging
• View graph, chart
• Assess application
• Configure alert
• View log & metric
gathered by Azure
Storage Analytic
AzCopy
• Download storage
account log
• Analyze using analysis
tool.

Storage Analytic Metric
Transaction
Metric
Summarize info about request
(read, write, delete, operation
status)
Capacity Metric
Capacity, container count, object
count
Troubleshooting with storage logging
Azure Storage
Analytic Logs
Details info
about request
to storage
Not enable by
default
Log version 1 or
2 (additional
field)

Thank You

Flash card architect storage infrastructure in azure

More Related Content

What's hot (17)

Similar to Flash card architect storage infrastructure in azure (20)

More from Yoong Seng Lai (16)

Recently uploaded (20)

Flash card architect storage infrastructure in azure