Zadara VMware Veam Hybrid Cloud
Zadara VMware Veam Hybrid Cloud
Steve Costigan
International Solutions Architect Director
Table of Contents
Introduction 2
Environment 2
Zadara® 2
The Challenge 3
Deployment Options 4
Summary 42
ENVIRONMENT
● Single Node VMware Software Defined Data Center in AWS using NSX-T
● Veeam Backup and Replication 9.5 Update 4 on SDDC with Zadara via NSX-T
● VMware SDDC Version 1.5+
● Zadara VPSA All Flash Array, Hybrid Array and Object Storage
● Zadara VSS Hardware Provider
ZADARA®
Zadara is agile, secure enterprise data storage built for hybrid IT. We help organizations
eliminate the cost and complexity traditionally associated with data storage, by providing
enterprise data storage solutions as a fully-managed service, with a 100%-uptime
guarantee and OpEx consumption-based pricing. Zadara uses industry-standard hardware
and patented Zadara software to deliver the power of enterprise-class data storage and
management — with the convenience of the cloud. Any data type. Any protocol. Any location.
Zadara is available via public clouds, managed service providers, data centers, colocation
partners, and on premises in customers' data centers.
Zadara’s storage services deliver high performance all-flash, flash cache-accelerated hard
disk, and object storage services with unique multi-tenant resource isolation. This ability to
deliver multiple storage tiers with consistent and predictable performance, even in shared
environments, is the perfect complement to Veeam Availability Suite and VMware Clouds.
2
Replication™ paired with the advanced monitoring and reporting of Veeam ONE™ for holistic
coverage of all workloads.
THE CHALLENGE
The objective is to create a complete, production-ready hybrid cloud environment that offers
high-performance, a familiar and proven operating environment, and complete data
backup, recovery, and replication capabilities. Achieving these ambitious goals with legacy
solutions would generally require extensive custom integration of multiple complex
packages. However, by leveraging Zadara Enterprise Storage-as-a-Service, VMware Cloud,
and Veeam Availability Suite, our goals can be achieved by a knowledgeable practitioner in
just a few hours.
Creating Highly Available Hybrid Clouds with Zadara, VMware, and Veeam 3
DEPLOYMENT OPTIONS
For testing purposes a single node SDDC VMC was defined in AWS London. Setup consists
of a single node with 10TB of vSAN SSD storage, split into:
● A vSAN Datastore for vCenter, NSX, and other VMware-provided management VMs
● A Workload Datastore for hosting user VM’s.
First the environment must be defined in the intended location, in this case AWS London.
There are several steps required for this, including defining the management network and
the AWS VPC to attach to. You also need access to your AWS environment with permissions
to launch a Cloud Formation script, so planning ahead is essential to a smooth deployment.
Once deployment has been completed, which usually takes 120 minutes from submitting the
request, you can manage the security setup for the VMC SDDC. This includes allowing
access to the vCenter environment via the public IP defined to it, enabling access from the
VPC EC2 instances, and enabling access via AWS Direct Connect, if applicable.
Once this is complete you can test accessibility to your VMC vCenter. At this point you may
want to enable Direct Connect to your VMware Cloud management network, this is where
the up-front planning will come in as it is imperative to avoid accidentally overlapping
address ranges. If accessing the VMware Cloud management network via Direct Connect,
configure the interfaces to be offered to the VMware-specific account supplied as part of
the VMware Cloud install.
4
This is where it gets interesting. In the current VMware Cloud on AWS, guest VMs are stored
on the SSD pool (10 TB raw per ESXi host, approximately 15 TB usable for a 3-node cluster).
This storage provides very good performance, but has two key shortcomings:
● Only one vSAN environment is supported. Therefore all backup storage traffic will
compete directly with production workloads. This is not good
● There is no option to provide multiple performance tiers to align application
requirements, performance, and cost
If we don’t want to have backup traffic impacting production and/or we want the ability to
provide applications with storage capacity that aligns with business requirements, what are
the options?
Prior to SDDC v1.5, we were limited by NSX-V functionality to backup applications that could
track and backup VMware environments. However, these solutions tend to require complex
configuration, particularly when implementing the best practice of automatic offsite backup
creation is a design point.
One possible approach would be to deploy a Linux VM, connected to EBS, and running SSH
and Perl to expose an NFS export. However this is complex and also creates a single point of
failure (the NFS export). Furthermore, going this route will mean that everything is going to
be located in the same Availability Zone (AZ) and Data Center, which is a suboptimal backup
strategy. Therefore, S3 must also be utilised for protection to meet the requirements, adding
yet another layer of complexity.
You could address availability concerns by keeping another copy in another AZ, but now you
are adding complexity; 2 Linux nodes, 2 sets of EBS. And this is going to need to be SSD as
the magnetic option has a 1TB limit. The backup costs are rising precipitously and
complexity is increasing geometrically, greatly increasing the risk of failure.
Fortunately, Veeam is a certified VMware Cloud backup solution and listed in the VMware
Cloud Marketplace, so we can use it in the VMware Cloud and/or the EC2 VPC, but we still
need to connect Veeam to appropriate storage. The simple solution is to present Zadara
iSCSI block or SMB NAS volumes to the EC2 instance. This way, even initial backups are
automatically stored outside of the AWS data center housing the VMware Cloud and
EC2 VPCs.
Creating Highly Available Hybrid Clouds with Zadara, VMware, and Veeam 5
With NSX-T, we now have an even simpler option; we can now use Zadara’s enterprise
storage-as-a-service to provide an iSCSI Block volume to a Veeam Backup Server within
VMware Cloud environment. This enables a backup repository which has high availability
built in, enhancing the availability and integrity of backups.
To further enhance data availability, the backups created on Zadara storage can be
replicated to any other Zadara virtual private array or, alternatively via Zadara Backup to
Object Storage, to any S3 / Swift compatible Object storage array including Zadara’s object
storage service or Amazon S3 itself.
In addition to the object storage service, Zadara storage also offers NAS with NFS and SMB
services. Either or both of these services can be used for primary data as well as backup. For
this project, we will implement the Active Directory-integrated SMB NAS service. Physics and
locking permitting, our service will be capable of serving the same data to applications and
users within our VMware Cloud, the EC2 VPC and on-premises.
Consider the possibilities! VDI running in AWS, accessing servers in VMware Cloud, with
data persisted on highly available shared storage, offering client-based recovery using the
standard “Previous Versions” capability built into every Windows client. And all the shared
data can be automatically protected with Zadara’s optional integrated McAfee antivirus.
Veeam and Zadara integrated into a hybrid cloud solution, data automatically backed up to
S3, set it forget it, but know it will be where you need it when you need it.
Need to support Windows and Linux? No problem. Support NFS clients with the same
replication capabilities as SMB. Support access to the same files with NFS and SMB. All
wrapped up with an integrated backup solution using Windows and Linux Agents from
Veeam.
The last piece of our project will be to support Windows Failover Server Clustering (WFSC).
For deployments limited to vSAN, there is good news and bad news. The good news is that
vSAN 6.7 introduced support for failover clusters via the iSCSI target Service, and the latest
VMware Cloud on AWS SDDC enables a shared VMDK capability. The bad news is that the
current offerings do not provide snapshots, let alone a Volume Shadow Copy Service (VSS)
provider to ensure application consistent snapshots.
6
Implementing hybrid cloud storage from Zadara, addresses these limitations and provides
additional availability options like remote mirroring and multi-zone HA, while enabling all of
the benefits of VMware Cloud on AWS and providing a consistent service, independent of
how (and where) your Zadara storage is physically deployed.
Creating Highly Available Hybrid Clouds with Zadara, VMware, and Veeam 7
Below is a typical top-level Zadara-to-SDDC-to-VPC configuration.
8
● Link to your AWS Account. Note that you must have Cloud formation rights in your
AWS Account
● Identify the VPC and subnet for your environment that you want to connect to your
VMC SDDC
● Once Connection has been successfully established, you will see the following
success message
Creating Highly Available Hybrid Clouds with Zadara, VMware, and Veeam 9
● Once All Networking information has been submitted and the request to create has
been completed you will see the following screens
10
● Upon completion you will see the following
● When logging into the VMware in AWS console, you will see the following for each
SDDC that you have defined
● View Details displays a summary and allows you to set the networking firewall rules
Creating Highly Available Hybrid Clouds with Zadara, VMware, and Veeam 11
● Networking and Security displays an overview of connections and how they are
related to various network environments
Note the Public IP is the Public IP of the vCenter mapped by VMware Cloud on AWS for
management. The management network is defined previously, during SDDC setup.
In the Security Tab you need to define access rules for management and access to vCenter.
By default, everything is denied. Create new objects for specific IP’s to enable access and
then grant access to vCenter, for instance:
At this point you can begin setting up the SDDC environment and begin the process of
requesting the Zadara virtual private array that we are going to connect.
Before requesting the Zadara storage service, we need to get the AWS Account ID for the
SDDC. Click on Direct Connect in the SDDC Networking and Security tab.
12
ZADARA STORAGE SERVICES DEPLOYMENT
For the next step, head over to https://manage.zadarastorage.com and request a new AWS
VPSA to be created in the AWS region containing the just-deployed SDDC. Alternatively, you
can sign in to your AWS Account and make the request via marketplace.
https://aws.amazon.com/marketplace/pp/B07K4WPL9X?qid=1564583550014&sr=0-
1&ref_=srh_res_product_title.
Complete the details in the signup form or once complete send an email to
[email protected]. Document the virtual array name, your account name, and the
AWS Account ID that you want the Direct Connect Virtual Interfaces (DXVI) offered to, as well
as the BGP ASN assigned to your environment.
Note: The AWS VPC Account ID and the AWS VMware Cloud Account ID’s are different. You
need to supply the VMware-supplied account details.
Once they have been offered, head back into the SDDC Direct Connect pane and accept
the 2 virtual interfaces.
After a few minutes you should see something similar to this, showing the learned and
advertised routes from the SDDC and the Zadara sides.
You will also need to configure rules to allow access between the VM Networks and the
VPC’s, the VM Network and the Direct Connect allowing specific services as required to
the VPSA Arrays
Creating Highly Available Hybrid Clouds with Zadara, VMware, and Veeam 13
Note: At this point you may need to ensure that traffic can flow via the VPC ENI to the
SDDC. Ensure that the rules applied allow two way communication between the EC2
instances and any configured VM’s in the compute gateway.
If your VPC also needs iSCSI, NFS, SMB storage from a Zadara VPSA Array, then
additional DXVIs will need to be provisioned – speak to your Zadara account manager
about this.
At this point you can either continue managing and provisioning the Zadara storage, or
you can schedule an “onboarding” session with Zadara support to help get you up
on running.
If we now log in to our VPSA you should see a Dashboard similar to this (this example is
using a Zadara all-flash array).
14
Or this if using a Zadara hybrid storage array:
The Servers Tab allows us to register new Clients and setup access to volumes. Here you
can see that we are presenting NFS volumes to ESX Servers as a datastore. We also have
some Windows clients defined in both the AWS VPC and the ESX Compute Network
via NSX-T.
Creating Highly Available Hybrid Clouds with Zadara, VMware, and Veeam 15
Servers can be registered manually or automatically via a setup script from the virtual
private array.
Below, we can see the range of volumes presented from our virtual private array to both EC2
instances, ESXi Hosts, and VM Guests, all from a single Zadara VPSA. Actual deployment and
capabilities may vary by region. Check with your Zadara team for capabilities in each
region.
Here you can see the two Zadara-presented NFS Datastores attached to the SDDC running
in VMware for AWS. As of this writing, mounting NFS volumes along with NSX-T is in
development / preview mode. Check out the VMware Cloud on AWS roadmap for more
details https://cloud.vmware.com/vmc-aws/roadmap.
There are a number of ways that virtual machines can be created or imported. VMs can be
imported from NFS mounts. They can be imported from Veeam backups that have been
replicated to and mounted by an EC2 instance, and subsequently copied to the vSAN SDDC
datastore. For environments that do not implement Veeam on the primary site, VMs can also
be imported from Zadara Backup to Object Storage backups stored on Zadara Object
Storage, Amazon S3, or Google Cloud. This option requires an ESXi environment, either
16
customer-owned or via a Zadara service partner, to mount native ESX Datastores prior to
restoring contained virtual machines.
This example virtual array is also serving Active Directory-integrated SMB Volumes to clients
with from AD controllers in EC2 and VM Guest instances. This capability is extremely useful
when deploying VMware Horizon 7 View to the SDDC environment. Deploying these AD-
integrated block devices enables roaming profiles and home directories to be centrally
stored and backed up. Add Zadara’s support for user and group quotas, optional integrated
antivirus protection, and full suite of replication capabilities (snapshots, local clones, remote
clones, and remote mirroring) and we have a hybrid cloud environment ideally suited for VDI
use cases.
This hybrid cloud storage configuration also enables Windows Clients in either the SDDC or
the VMware Cloud to make use of the Windows Previous Versions restores, natively, from
within the standard windows explorer tools for self service data recovery.
Creating Highly Available Hybrid Clouds with Zadara, VMware, and Veeam 17
Combining the snapshots with replication enables a number of advanced options
features such as:
● Data protection via geographic distribution
● On-premises-to-cloud disaster recovery
● Cloud-to-on-premises disaster recovery
● Populating test environments with production data
● Replicating between different types of storage pools, e.g. from accelerated hard disk
to all-flash pools
● Replicating production to enable rapid testing with independent copies of
production data
18
In particular, Veeam’s capabilities for backing up VMware Cloud on AWS, combined
with straightforward integration for Zadara’s enterprise storage-as-a-service, provides
provides an excellent path for delivering hybrid cloud-aware enterprise grade data
availability services.
More specifically, creating a Veeam backup infrastructure running in the SDDC, with storage
presented from Zadara to the backup servers as iSCSI Block or SMB Shares, enables a
backup schedule to be defined that places both second and third data copies outside of the
all-flash vSAN environment. Leveraging storage that is independent of the vSAN production
datastores offers three key advantages:
● Backup storage I/O will not compete with production workloads
● Higher capacity storage pools enable longer retention time
● Leveraging hard disk capacity reduces backup storage costs
First we create the Veeam Windows Server as VM Guest, install the required updates, start
the iSCSI service and register the hosts with the Zadara VPSA.
Then we map an iSCSI volume that will be used to store the VSAN backups, format it and
prepare to install the Veeam backup software. Download the Veeam 9.5 Update 4 image,
mount it and begin the installation.
Creating Highly Available Hybrid Clouds with Zadara, VMware, and Veeam 19
Outside of the VMware Cloud on AWS SDDC we could create 2 types of repository
disks, 1 running from VMFS via an NFS, or Block mounted Datastore and a direct iSCSI
Volume to a Zadara Block Volume.
20
Creating Highly Available Hybrid Clouds with Zadara, VMware, and Veeam 21
Enable and setup vPower NFS so that change blocks on instant recovered VM’s can be
storedand tracked, a separate volume can be used but a new folder was used for
this environment.
22
Creating Highly Available Hybrid Clouds with Zadara, VMware, and Veeam 23
VEEAM BACKUP JOB CREATION
Create a Backup Job Below we will see the details of a created Job to backup an AD Server.
24
Creating Highly Available Hybrid Clouds with Zadara, VMware, and Veeam 25
26
Job statistics from Backup Job Incremental
Creating Highly Available Hybrid Clouds with Zadara, VMware, and Veeam 27
Veeam Backup Files showing Full and Incremental Files:
Veeam 9.5 update 4 provides the ability to add a Scale-Out Backup Repository (SOBR)
utilising AWS S3 and qualified S3 compatible Object Storage. This offers the option of
combining Zadara as an offsite, off-cloud, primary backup location and utilising AWS S3 for
a multiple Availability Zone(local or geographically remote) target. This provides added
capabilities and options for both On-Premises and Public Cloud solutions for long term data
retention and disaster recovery options.
Users can also make use of their complete Zadara platform to provide the Object Scale Out
Backup Repository as well as the Primary Data Copy see “Veeam - Configure VPSA Object
Storage as an Object Storage Repository” (https://support.zadarastorage.com/hc/en-
us/articles/360027853491-Veeam-Configure-VPSA-Object-Storage-as-an-Object-Storage-
Repository) for best practices.
Zadara and Veeam provide joint support for using Zadara’s object storage service via
Zadara’s S3 compatibility. Please see the interim compatibility list at
https://forums.veeam.com/object-storage-f52/unoffizial-compatibility-list-for-veeam-cloud-
tier-t56956.html.
28
The following diagram illustrates the value that integrating Veeam and Zadara provides; a
multiple copy, multi-site, all tier storage and backup solution.
● Ransomware protection
● Instant population of test environments with complete sets of application data
● Verification of backed up data prior to offsite transfer
Creating Highly Available Hybrid Clouds with Zadara, VMware, and Veeam 29
ZADARA VSS HW PROVIDER DEPLOYMENT:
The Zadara VSS Hardware Provider can be downloaded from the following URL
https://zadarastorage-software.s3.amazonaws.com/ZadaraHardwareProvider_x64.msi for
64bit Windows Clients and
https://zadarastoragesoftware.s3.amazonaws.com/ZadaraHardwareProvider_x86.msi for
32bit Windows Clients
Once downloaded and installed you will need to configure access to your VPSA
Array API:
● Create a specific user within the VPSA Array (refer to user documentation)
● Login as the user and present the IP address of the VPSA and the access key for your
user with permissions to create snapshots and clones
30
Microsoft has released a number of CLI VSS tools over the years here we will use vssadmin
and DiskShadow to demonstrate some of the capabilities. Open a command prompt as an
Administrator and execute “vssadmin list providers” You should see something like the
below image showing the ZadaraVssProvider.
Creating Highly Available Hybrid Clouds with Zadara, VMware, and Veeam 31
Check the Volumes with VSSAdmin, Volumes E: and F: are iSCSI presented Zadara Volumes
of 10Gb and 1TB respectively.
32
Before Creating the VSS Snapshot there are no Snapshots or Clones of this volume.
You may see an error on VSS Object Creation in some Windows Installations. This is a
known Windows error and does not interfere with the VSS snapshot creation. It is a
Microsoft BCD error (see this technote https://support.microsoft.com/en-
gb/help/3025158/diskshadow-error-when-you-try-to-create-a-vss-snapshot-in-windows-
serv).
Creating Highly Available Hybrid Clouds with Zadara, VMware, and Veeam 33
Here are the set of commands used to create the VSS Aware Snapshot as a
Transportable Snapshot.
Here is the Metadata created during snapshot creation on the SMBTest NAS Share.
Now in our VPSA Array we can see the creation of the snapshot, but there is no clone at this
point. It is just an application consistent point in time.
34
On our Test / Dev, Backup Server we can now create a Clone and access the data.
Let’s use DiskShadow this time to list the providers instead of VSSAdmin.
Now import the Metadata from the Cab file into the VM with Diskshadow.
Creating Highly Available Hybrid Clouds with Zadara, VMware, and Veeam 35
Check the Shadow Data has imported that this has come from our other Server.
36
Now we need to expose this ShadowCopy Volume to the OS
Creating Highly Available Hybrid Clouds with Zadara, VMware, and Veeam 37
If we now check the Windows Disk Manager we can see the disk has mounted.
Windows Explorer and we can see the files present at the time of the snapshot on the
original host.
38
Back to our VPSA Array GUI and we can see we have a new volume with the name of the
snapshot, starting with its ID, some unique identifier, and ending in “.clone”
A nice feature of DiskShadow is you can use an input file to automate the above commands
with a scripting language like PowerShell. This will work for any VSS Aware Application, SQL
Server, Exchange, Oracle etc. on a Windows Server.
For other applications such as Linux you can use application tools such as RMAN to place
the Database into hot standby mode and use the Zadara API or the Python module zadarapy
to do the automation.
Creating Highly Available Hybrid Clouds with Zadara, VMware, and Veeam 39
Here is a simple script for the application to create a transportable snapshot:
end backup
#end of myVSSCreateScript.txt
#end myVSSImportScript.txt
Note: A clone presented to a Windows host is treated as read-only by that Windows host,
even though it is read / write as far as the Zadara storage is concerned. To gain write
access from Windows, break VSS snapshot with the following command before presenting
the volume.
40
Once the volume is no longer required you can unpresent it and remove the shadow.
This will remove the volume from the VPSA also, unless the volume is also mapped to
another host.
#end myRemoveVSSClone.txt
SUMMARY
In this document, we have demonstrated deploying Zadara enterprise storage-as-a-service
with VMware Cloud on AWS can deliver a simple-to-operate and highly available hybrid
cloud. By combining the flexibility and performance of Zadara with the native hybrid cloud
capabilities of VMware Cloud, we enable simplified workload migrations between VMware
Clouds, wherever located, without modifying applications and without compromising
security or performance.
Having established our “basic” hybrid cloud, we then extended it, adding three
key capabilities:
● Enterprise data protection and availability with Veeam Availability Suite 9.5
● VDI, including Windows-native self-service data recovery, with VMware
Horizon 7 View
● High availability for Windows enterprise applications with Windows Failover Server
Cluster and application-consistent snapshots
Creating Highly Available Hybrid Clouds with Zadara, VMware, and Veeam 41