Optimize Storage Compute Platforms in Vmware Vsphere Environments Best Practices Guide
Optimize Storage Compute Platforms in Vmware Vsphere Environments Best Practices Guide
November 2021
Feedback
Hitachi Vantara welcomes your feedback. Please share your thoughts by sending an email message to
[email protected]. To assist the routing of this message, use the paper number in the subject and the title of
this white paper in the text.
Revision History
Revision Changes Date
Hitachi Vantara LLC, a subsidiary of Hitachi, Ltd., provides various datacenter infrastructure components to enable IT
environments to support a VMware ecosystem. This includes midrange and enterprise storage, converged, and
hyperconverged infrastructure as well as a suite of software and software integrations to enable a robust automated
operational environment.
This document outlines most of the best practices to implement in a VMware server virtualization, desktop, or cloud
environment with Hitachi storage or a converged Hitachi Unified Compute Platform (UCP). This includes the associated
software integrations into various VMware vCenter and vRealize management stacks. These aid in building a VMware
environment that provides the performance, scalability, reliability, usability, resilience, and recoverability expected when
paired with Hitachi products.
Hitachi is an Elite Partner in VMware’s Technology Alliance Partner program, a participant in VMware Ready Partner
programs for Storage Infrastructure Services, and a Global OEM partner. Together, Hitachi and VMware are committed to
providing innovative, business-enabling technology, with end-to-end virtualization solutions for the datacenter.
These best practices cover the Hitachi storage and converged products listed in Table 1, “Hitachi Storage and Converged
Systems,” on page 2.
1
TABLE 1. HITACHI STORAGE AND CONVERGED SYSTEMS
Hardware Product
Storage Hitachi Virtual Storage Platform (VSP) 5000 series
Cisco and Hitachi Adaptive Solutions for SAP HANA Tailored Data
Center Integration
Some of the Hitachi software products covered by these best practices are listed in Table 2, “Hitachi Software, Plugin, and
Adapter Products,” on page 3.
2
TABLE 2. HITACHI SOFTWARE, PLUGIN, AND ADAPTER PRODUCTS
Software Product
Hitachi Adapters, Plugins and Hitachi Storage Provider for VMware vCenter (VASA)
software for VMware
Ecosystem Hitachi Infrastructure Management Pack for VMware vRealize
Operations (vROPS)
Hitachi Storage Content Pack for VMware vRealize Log Insight (vRLI)
3
Note — Testing to develop these best practices was in a lab environment. Many things affect production environments
beyond prediction or duplication in a lab environment. Follow the recommended practice of conducting proof-of-
concept testing for acceptable results in a non-production, isolated test environment that otherwise matches your
production environment before your production implementation of this solution.
Documentation on all of these Hitachi and VMware software integrations available on this VMware page from Hitachi
Vantara Support. Download the integrations from VMware Adapters on Hitachi Vantara Support (user credentials required)
or Hitachi Vantara on the VMware Solution Exchange (no credentials required).
For more information, see VMware vSphere Virtual Volumes (vVols) with Hitachi Virtual Storage Platform Quick Start and
Reference Guide.
4
Hitachi Infrastructure Management Pack for VMware vRealize Operations
Hitachi Infrastructure Management Pack for VMware vRealize Operations (vROPS) (formerly Hitachi Storage Management
Pack for VMware vRealize Operations) integrates metrics and alerts from physical and virtual layers to help you manage the
health, capacity and performance of your Hitachi storage or converged infrastructure deployments in VMware
environments. It significantly enables efficient resource utilization and proactive troubleshooting to reduce operational
costs leveraging the provided dashboards, metrics, and correlated alerts. For more information, see the Infrastructure
Management Pack for VMware vRealize Operations user guide.
5
Ops Center Protector Adapter for VMware Site Recovery Manager
Hitachi Data Instance Director Adapter for VMware Site Recover Manager provides a higher level of automation for
configuration of local and remote replication relationships between primary and secondary systems. This adapter is like the
adapter referenced above. This adapter is compatible with Hitachi Ops Center Protector environments that manage all the
pausing, swapping, and resuming of the associated replication pairs that VMware vCenter Site Recovery Manager may
require. Deploy this adapter independently from the adapter referenced above. For more information, see Data Instance
Director Adapter for VMware Site Recovery Manager.
Unified Compute Platform Advisor is used to discover and provision servers initially, and later to manage the compute
nodes:
These performance enhancements move the I/O load from the dependent VMware vCenter host platform into the storage
controller. By offloading storage related operations off to the storage subsystem, it speeds up the datastore and VMDK
provisioning operations. This frees virtualization management for more critical tasks.
When used with VMware vSphere 5.x, 6.x, and 7.x, Hitachi storage supports the following API primitives:
Full copy — This primitive enables the storage system to make full copies of data within the storage system without
having the VMware ESXi host read and write the data.
Block zeroing — This primitive enables storage systems to zero out many blocks to speed provisioning of virtual
machines.
6
Hardware-assisted locking — This primitive provides an alternative means to protect the metadata for VMFS cluster file
systems, thereby improving the scalability of large VMware ESXi host farms sharing a datastore.
Thin provisioning stun — This primitive enables the storage system to notify the VMware ESXi host when thin
provisioned volumes reach certain capacity utilization threshold. When enabled, this allows the ESXi host to take
preventive measures to maintain virtual machine integrity.
UNMAP — This primitive enables a VMware ESXi host to inform the Hitachi storage array that space can be reclaimed
that previously had been occupied by a virtual machine that has been migrated to another datastore or deleted.
7
Hitachi SAN and VMware Configuration Best Practices
A well-designed SAN must be reliable and scalable and recover quickly in the event of a single device failure. Also, a well-
designed SAN grows easily as the demands of the infrastructure that it serves increases. The focus of this best practice
guide is on environments that leverage SAN-based datastores. If you use Hitachi NFS datastores, consult Hitachi NAS
Platform Best Practices Guide for NFS with VMware vSphere (MK-92HNAS028-01 or later, PDF).
Hitachi storage uses Hitachi Storage Virtualization Operating System RF (SVOS RF). Find general documents in Storage
Virtualization Operating System covering volume management, security replication. Following are specific advice for
VMware environments using Storage Virtualization Operating System. This guide covers VMware vSphere 6.x and 7.0
environments at time of publication.
Virtual Volumes (vVols) covered in detail in “Hitachi Storage (VASA) Provider Enabling Virtual Volumes (vVols)” on page 12.
Follow the guidance covered in VMware vSphere Virtual Volumes (vVols) with Hitachi Virtual Storage Platform Quick Start
and Reference Guide.
LUN Size
The following lists the current maximum LUN/datastore size for VMware vSphere and Hitachi Storage:
The maximum LUN size for VMware vSphere 6.x or 7.x is 64 TB.
The maximum LUN size for Hitachi Virtual Storage Platform F series or VSP G series is 256 TB with replication.
The LUN must be within a dynamic provisioning pool.
The maximum VMDK size is 62 TB (vVol-VMDK or RDM)
Using multiple smaller sized LUNs tend to provide higher aggregated I/O performance by reducing the concentration of a
storage processor (MPB). It also reduces the recovery time in the event of a disaster. Take these points into consideration if
using with larger LUNs. In some environments, the convenience of using larger LUNs might outweigh the relatively minor
performance disadvantage.
Prior to Hitachi Virtual Storage Platform E series, VSP F series, and VSP G series systems, the maximum supported LUN size
was limited to 4 TB because of storage replication capability. With current Hitachi Virtual Storage Platform, this limitation
has been removed. Keep in mind that recovery is typically quicker with smaller LUNs. So, use the appropriate size that
maximizes usage of MPB resources per LUN for workload. Use the VMware integrated adapters or plugins Hitachi Vantara
provides, such as vSphere Plugin, Microsoft PowerShell cmdlets or VMware vRealize Orchestrator workflows to automate
datastore and LUN management.
8
Thin-Provisioned VMDKs on a Thin-Provisioned LUNs from Dynamic ProvisioningPool
Thin provisioned VMDKs on thin provisioned LUNs have become a common storage provisioning configuration for
virtualized environments. While EagerZeroThick VMDKs have typically seen better latency performance in older vSphere
releases (that is, releases older than vSphere 5), the performance gap between thin VMDK and thick VMDK is now
insignificant and you get added benefits with in-guest UNMAP for better space efficiency with thin. In vVols, thin
provisioned VMDK (vVol) is the norm and it performs even better than thin VMDK on VMFS as no zeroing is required when
allocating blocks (Thin vVols are the new EZT !). Generally, start with thin VMDK on VMFS or vVols datastores. The only
exception where you might consider migrating to EZT disks is if you have performance sensitive heavy write VM/container
workloads where you can potentially see low single digit % performance improvement for those initial writes that might
not be noticeable to your app.
In the VSP storage array with Hitachi Dynamic Provisioning, it is also quite common to provision thin LUNs with less
physical storage capacity (as opposed to fully allocated LUNs) However, monitor storage usage closely to avoid running
out of physical storage capacity. The following are some storage management and monitoring recommendations:
Hitachi Infrastructure Management Pack for VMware vRealize Operations provides dashboards and alerting capability
for monitoring physical and logical storage capacity
Enable automatic UNMAP with VMFS 6 (scripted UNMAP command with VMFS 5) to maintain higher capacity efficiency
LUN Distribution
The general recommendation is to distribute LUNs and workloads so that each host has 2-8 paths to each LDEV. This
prevents workload pressure on a small set target ports to become a potential performance bottleneck.
It is prudent to isolate production, and critical systems to dedicated ports to avoid contention from other hosts workloads.
However, presenting the same LUN to too many target ports could also introduce additional problems with slower error
recovery.
Each host bus adapter physical port (HBA) should only see one instance of each LUN
The number of paths should typically not exceed the number of HBA ports for better reliability and recovery
Two to four paths to each LUN provides the optimal performance for most workload environments
See Recommended Multipath Settings for Hitachi Storage knowledge base article for more information about LUN
instances.
In certain circumstances, increasing the queue depth value may increase overall I/O throughput. For example, a LUN
hosting as a target for virtual machine backups might require higher throughput during the backup window. Make sure to
monitor storage processor usage carefully for queue depth changes.
Slower hosts with read-intensive workloads may request more data than they can remove from the fabric in a timely
manner. Lowering queue depth value can be an effective control mechanism to limit slower hosts.
For a VMware vSphere protocol endpoint (PE) configured to enable virtual volumes (vVols) from Hitachi storage, set a
higher queue depth value, such as 128.
9
Host Group and Host Mode Options
To grant a host access to an LDEV, assign a logical unit number (LUN) within a host group. These are the settings and LUN
mapping for host group configurations.
Port security — Set the port security to Enable. This allows multiple host groups on the Fibre Channel port.
Fabric — Set fabric to ON. This allows connection to a Fibre Channel switch or director.
Connection Type — Set the connection type to P-to-P. This allows a point-to-point connection to a Fibre Channel switch
or director. Loop Attachment is deprecated and no longer supported on 16 Gb/s and 32 Gb/s storage channel ports.
Hitachi recommends that you apply the same configuration to a port in cluster 1 as to a port in cluster 2 in the same location.
For example, if you create a host group for a host on port CL1-A, also create a host group for that host on port CL2-A.
However, in a cluster environment, this approach can be an administrative challenge because keeping track of which WWNs
for ESXi hosts are in a cluster can be difficult. When multiple ESXi hosts need to access the same LDEV for clustering
purposes, the LDEV must be added to each host group.
For convenience and where granular control is not essential, create host groups with clustering in mind. Place all the
WWNs for the clustered ESXi hosts in a single host group. This ensures that when adding LDEVs to the host group, all ESXi
hosts see the same LUNs. This creates consistency with LUN presentation across all hosts.
Host Mode
21 [VMware Extension]
Host Mode Options:
Enable 54-(VAAI) Support Option for the EXTENDED COPY command (redundancy optional).
Enable 63-(VAAI) Support option for vStorage APIs based on T10 standards (This includes extended copy via T10
which is why HMO 54 is now redundant or ignored for all SVOS releases ).
Enable 114-(Auto-UNMAP) Automatic asynchronous reclamation on ESXi 6.5 or later (vSphere 7.0 U2 and later
supports granular unmap greater than 1 MB so this host mode option is no longer needed with that release or
higher).
10
When using VMware Virtual Volumes (vVols) environment on Hitachi storage, use the same options as above plus the
following:
Disable the custom Hitachi VAAI plugin claimrules on ESXi hosts, if present, so that VAAI T10 is exclusively used. Review
the quick start guide for vVols, VMware vSphere Virtual Volumes (vVols) with Hitachi Virtual Storage Platform Quick
Start and Reference Guide, on how to manage VAAI claimrules on ESXi hosts. This custom plugin claimrules are no
longer being used and are being removed from future versions of VMware vSphere.
VMware vSphere Storage APIs Array Integration (VAAI) — Atomic Test and Set (ATS)
A change in the VMFS heartbeat update method was introduced in VMware VMFS 5, and this optimization results in a
significant increase in the volume of ATS commands the ESXi kernel issues to the storage system and causes increased load
on the storage system. Under certain circumstances, VMFS heartbeat using ATS may fail with false ATS miscompare events.
This causes the ESXi kernel to verify again its access to VMFS datastores. This leads to “Lost access to datastore”
messages.
The resolution of this issue is implemented in VMFS 6. The following setting is recommended for a VMware vSphere 6.5
environment.
Set ATS heartbeat OFF for vSphere 6.0 or later with VMFS5.
Keep default ATS heartbeat ON for vSphere 6.0 or later with VMFS6 without global-active device configured.
Set ATS heartbeat OFF for vSphere 6.0 or later with VMFS6 with global-active device configured.
Refer to ESXi host loses connectivity to a VMFS3 and VMFS5 datastore (2113956) for more details and how to turn off ATS.
Zoning
Use zoning to enable access control in a SAN environment. Through zoning, a SAN administrator can configure which HBA
WWPNs on the VMware ESXi host can connect to which WWPNs on the Hitachi storage processors.
The VMware ESXi host port in the Fibre Channel HBA is referred to as the initiator. The storage processor port in the Hitachi
storage array is referred to as the target.
You can break zoning down into the following different configurations:
Single Initiator to Single Target (SI-ST) Zoning — This configuration allows one initiator to be zoned to only one target.
This configuration is the most resilient configuration, as traffic originating from another Initiator on the SAN will have
less impact than the initiator in this zone.
Brocade Peer Zoning – This configuration allows a single zone to provide a Principal–Pupil relationship where all pupils
can communicate with the principal but no with each other. This provides the same zone-security as SI-ST zoning but
with the administrative benefit of a reduction of number of zones. This is the preferred configuration in a Brocade fabric
configuration.
Cisco Smart Zoning – This implementation is preferred in a Cisco environment where NX-OS can eliminate initiator to
initiator and target to target communication.
Single Initiator to Multiple Target (SI-MT) Zoning — This configuration allows one initiator to be zoned to multiple
targets in a single zone.
Multi Initiator Zoning — This configuration is never recommended. This configuration allows multiple initiators to be
zoned to multiple targets in a single zone. This exposes all initiators and targets to all traffic in the zone. Events such as
a malfunctioning HBA could affect all initiators and targets in the zone and either negatively affect performance for all
or bring down the Fibre Channel network completely.
11
Hitachi generally recommends the following:
For utmost availability with slightly higher administrative cost, Hitachi recommends SI-ST zoning. Brocade Peer Zoning
and Cisco Smart Zoning is supported to reduce admin burden.
Each HBA port should only see one instance of each LUN. This is primarily based on years of experience with fabrics
and to avoid potential availability issues where host HBA ports can be overrun leading to performance issues and error
recovery with fabric path issues (transient or otherwise) is faster and less impactful to hosts.
See Recommended Multipath Settings for Hitachi Storage knowledge base article for more information.
Optionally, do the following:
Use SI-MT with Brocade Peer Zoning or Cisco Smart Zoning and follow same LUN presentation recommendation above.
Regarding SI-MT, an example to use is provided within Cisco and Hitachi Adaptive Solutions for Converged
Infrastructure and Cisco and Hitachi Adaptive Solutions for SAP HANA Tailored Data Center Integration.
Zoning is configured as SI-MT with Cisco Smart Zoning to optimize traffic intended to be specific to the initiator (Cisco UCS
host vHBA) and the targets (Hitachi Virtual Storage Platform controller ports).
Using SI-MT zoning provides reduced administrative overhead versus configuring traditional SI-ST zoning, and results in
the same SAN switching efficiency when configured with Smart Zoning. Refer to the Cisco and Hitachi Adaptive Solutions
for Converged Infrastructure Design Guide for more details.
Multipathing
Multipathing allows a VMware ESXi host to use more than one physical path between the ESXi host and the storage array.
Multipathing provides load balancing. This is the process of distributing I/O across multiple physical paths, to reduce or
remove potential bottlenecks. Multipathing also provides redundancy and fault tolerance in case of a failure of any element
in the SAN network, such as an adapter, switch, or cable. The ESXi host can switch to another physical path that does not
use the failed component. This process of path switching to avoid failed components is known as path failover.
To support path switching with a Fibre Channel SAN, the ESXi host typically has two or more HBAs available from which the
storage array can be reached. It also has full fault tolerance that uses two or more switches. Additionally, for full fault
tolerance, two storage processors on Hitachi Storage arrays should be utilized so that the HBA can use a different path to
reach the disk array.
Available multipathing policies supported by ESXi hosts are Round Robin, Most Recently Used, Fixed, and Hitachi Dynamic
Link Manager.
Hitachi recommends using the Round Robin Multipathing PSP policy (VMW_PSP_RR) and use SATP default of active-active
(VMW_SATP_DEFAULT_AA). In a global-active device configuration, Round Robin Multipathing PSP and using ALUA SATP
(VMW_SATP_ALUA) are recommended options. This multipathing policy takes advantage of all available paths and
bandwidth. Taking advantage of all available paths assures maximum performance from the SAN infrastructure. Note, With
vSphere 6.7U1 and vSphere 6.5 P03 or later, the round robin multipathing policy became the default setting as part of the
SATP claimrules for Hitachi Storage. See this blog post for more details.
In a global-active device configuration without ALUA configured, Fixed policy is preferred PSP to ensure writes are sent to
preferred side.
As part of VMware ESXi Round Robin Path Selection Plug-in (PSP), there is an I/O quantity value when a path change is
triggered that is known as the limit. After reaching that I/O limit, the PSP selects the next path in the list.
The default I/O limit is 1000 but can be adjusted if needed to improve performance. Specifically, it can be adjusted to reduce
latency seen by the ESXi host when the storage system does not see latency.
12
The general recommendation for the PSP limit is to continue to use the default value of 1000 in typical VMware's mixed
environment with multiple ESXi hosts with multiple datastores. It has been observed that a value of 20 provides potentially
the optimum value for additional 3-5% latency improvement and potentially reducing path error detection. See this blog
post for more details.
Round Robin (VMware) — This policy sends a set number of I/O down the first available path, then sends the same set
number of I/O down the next available path. This repeats through all available paths, and then starts over again and
repeats. If a path becomes unavailable, it is skipped over until it becomes available again.
Most Recently Used (VMware) — This policy uses the last successfully used path. If the last successfully used path is
not available, then path failover occurs, and the next available path is used. This new path continues to be used until it is
no longer available, even if the previously used path becomes available again.
Fixed (VMware) — This policy has a preferred path that can be set by the VMware vCenter administrator. This path is
used until it becomes unavailable. Then, it fails over to the next available path until it becomes unavailable. In which
case, the path fails over to the next available path, or until the preferred path becomes available again. If the preferred
path does become available again, then the system fails back to the preferred path.
Hitachi Dynamic Link Manager — VMware ESXi also supports third party path selection policies. Hitachi Dynamic Link
Manager is Hitachi's multipathing software that integrates with global-active device on Hitachi Virtual Storage
Platform and Hitachi High Availability Manager to provide load balancing and path failover capabilities for servers.
In a VMware vSphere environment, the ESXi hosts should have two or more HBA ports. Allocate at least one HBA port for
each Fibre Channel fabric. Not only will this allow for greater I/O throughput on the SAN as more paths are available when
using the round robin (VMware) multipathing policy, multiple HBAs also allow for redundancy and greater reliability in case
of a component failure.
Each VMware ESXi host in a cluster should have an equal number of connections to each Fibre Chanel switch. Each Hitachi
Storage array should have an equal number of connections from each storage processor to each switch. The example of this
can be found in UCP CI for VMware vSphere Reference Architecture Guide. See “Configure Storage for Fibre Channel SAN”
in that document.
This SAN Fibre Channel switch configuration ensures that a single switch failure will not leave an ESXi host unable to
connect to a datastore, unable to continue running the virtual machines on those datastores.
It is recommended that the Fibre Channel switches not be up-linked to each other, creating separate Fibre Channel
networks. This ensures that conditions on a Fibre Channel network do not affect traffic on another Fibre Channel network,
such as would happen with a malfunctioning HBA. This helps ensure system reliability.
13
In return, in certain environments, VASA can deliver virtual machine storage requirements from vCenter Server to a storage
entity and ensure that the storage layer meets the requirements.
Read this first: VMware vSphere Virtual Volumes (vVols) with Hitachi Virtual Storage Platform Quick Start and
Reference Guide to successfully enable a vVol environment on Hitachi storage.
On the Hitachi storage system, the protocol endpoint (PE) is an assigned logical unit (ALU). An ALU must be assigned to
all VMware ESXi hosts in order to access vVols. A single ALU is all that is required for all ESXi hosts (up to a vSphere
maximum of about 16,000 vVols per PE). This PE is mapped to 4 or more storage ports. Multiple ALUs can be deployed if
required.
The following figure shows the steps to create an ALU in Hitachi Storage Navigator. This is a necessary pre-requisite
and best practice to create at least one ALU prior to VASA provider deployment.
14
vVols can share an existing pool or pools and resource groups with VMFS datastores or have dedicated pools and
resource groups. The best practice to date has been to create a separate resource group and pool which can share an
underlying parity groups or use dedicated parity group if available
Communication from the VASA provider to Hitachi storage for vVols operations is using the service processor (SVP).
You can deploy the SVP as a preconfigured physical 1U node (or nodes) or installed as a virtual machine appliance. See
System Management with SVP for more information about SVP installation options.
To ensure the high availability of vVol out-of-band management operations, treat the VASA appliance like availability
deployment modes used for vCenter appliance (VCSA) or NSX-T appliance. The VASA provider from Hitachi supports
the following availability features:
VMware vSphere Fault Tolerance (FT)
VMware vSphere High Availability (HA)
The VASA provider from VMware also supports monitoring of its application service level under the VMware vSphere
High Availability configuration. By enabling the monitoring of the virtual machine and application option within VMware
vSphere High Availability, the VASA provider will automatically restart if the VASA provider service stops.
When Storage Provider for VMware vCenter or SVP becomes unavailable, only storage management control operations
related to the virtual volume are affected, such as clone virtual machines, snapshot operations, and power on
operations. This out-of-band communication does not affect virtual machine I/O, as the I/O flows through the fabric
data path using protocol endpoints.
For example, the ability to have multiple vCenter servers that may be allocated for different groups or workloads to connect
and use single or multiple shared storage array or arrays using a single managed VASA provider. When deployed with
VMware Cloud Foundation, you typical deploy the VASA provider OVA from Hitachi in the management domain. Each
workload domain vCenter can register to that single VASA Provider.
If you want to use the same storage system with multiple VASA providers (such as Production and Dev/Test environments
with separate VASA providers), then best practice is you must create or use separate resource groups for each VASA
provider to avoid any interaction overlap.
Hitachi’s tag-based storage policy enablement can be used with only one vCenter server.
Policy based management is premised on the fact that storage exposes a set of capabilities. That is, a resource that
provides Tier 1 IOPS and encryption and remote replication services. Virtual machine administrators define virtual machine
storage policies policy for virtual machine producers selecting one or more of these capabilities and values that are
exposed by the VASA provider. When a virtual machine is provisioned and assigned a virtual machine storage policy,
vCenter asks the VASA provider to create that virtual machine and its VMDKs in storage resources or services that match
that virtual machine storage policy.
15
The following figures show an example of a storage capability profile definition by VASA provider in the web user
interface, an example virtual machine storage policy definition by VMware vSphere Web Client for both placement and
replication capabilities, and the visibility of those vVols and their associated storage policy assignment.
16
Note that replication services for Hitachi VASA Provider are enabled with Ops Center Protector
Hitachi VP Hitachi VP
Capacity
When you implement VMware vSphere Virtual Volumes (vVols), the legacy systems inherit the benefits of storage policy
based management (SPBM) and VASA or vVol implementation.
To enable data tiering between the system to enable use cases where virtual machines or data will age in importance over
time, this is best practice:
1. Create one vVol datastore (storage container) that abstracts the two or multiple storage systems into one vVol
datastore using Hitachi Data Tiering (HDT) pooling capabilities.
2. Create a VMware vCenter virtual machine storage policy with capabilities of Tier 3 IOPS+latency for these virtual
machine services.
3. Review the capability schema in the VASA web user interface to see how VASA maps storage policies to tiers created
with Hitachi Dynamic Tiering.
17
When you assign this policy to these aging or less relevant virtual machines, the VASA provider will detect that policy change
and automatically tier those warm or cold virtual machines or VMDKs to the lowest legacy tier in this configuration. This frees
up that tier 1 for the next revenue generating app.
Hitachi Storage (VASA) Provider Enabling Tag-based Storage Policy for VMFS
Datastores
The tag-based storage policy provides similar outcome for VMFS datastores as storage policy-based management (SPBM)
does for VMware Virtual Volumes.
You can set a storage capability profile on the pool that is serving the VMFS datastores or you can customize the storage
capability profile of an individual LUN. Hitachi Virtual Storage Platform automatically tags the datastores in VMware
vCenter for existing and new datastores and these will appear in vCenter as tags.
Like SPBM for VMware vVols, you can create a virtual machine storage policy using tags to allow virtual machine producers
to find the right datastore that matches their requirements. It also allows you to create custom capabilities (such as rack
location, availability zone) to augment the base capabilities provided.
18
Clustered VMDK with vSphere 7.0+
For details on setting up and using Clustered VMDK (or Shared VMDK) with Hitachi Virtual Storage Platform, see our
detailed blog at https://community.hitachivantara.com/s/article/Setting-up-Windows-Server-Failover-Cluster-on-
vSphere-7-with-Hitachi-VSP.
iSCSI
This section describes volume provisioning using iSCSI.
iSCSI Provisioning
iSCSI initiators and targets use TCP to create relationships called sessions. The initiator sees one logical connection to the
target. An iSCSI session might also contain multiple logical connections.
19
From a VMware vSphere host perspective, these sessions might also be thought of in term of paths between the initiator
and target. Having multiple connections per session enables bandwidth aggregation and can also provide load balancing.
Although vSphere does not support multiple connections per session, by configuring multiple sessions, you can configure
load balancing and ensure path redundancy. See VMware vSphere Storage Guide 6.7 (PDF) for more details.
Multipathing plug-ins do not have direct access to physical NICs on your host. So, for this setup, you first need to connect
each physical NIC to a separate VMkernel port. You then associate all VMkernel ports with the software iSCSI initiator using
a port binding technique. As a result, each VMkernel port connected to a separate NIC becomes a different path that the
iSCSI storage stack and its storage-aware multipathing plug-ins can use. Refer to Best Practices For Running VMware
vSphere On iSCSI for more information.
UNMAP
In VMware vSphere 6.5, automatic UNMAP was introduced. It automatically issues the UNMAP command to release free
storage space in background on thin-provisioned storage arrays that support UNMAP operations.
Use Thin provisioned VMDKs backed with thin provisioned LDEVs/LUs or vVols datastore
VMFS 6 datastores
In-Guest UNMAP support:
Linux guest OS with hardware version 13 or later to present SCSI-4
Windows 2012R2 OS with hardware version 11 or later
In-Guest automated UNMAP also supported for VMware vVol datastores.
vSphere ESXi 6.5 P1 or U1 or later (VMware ESXi 6.5.0 build-4564106 used internally)
Storage: VSP E series, VSP G series, VSP F series, or VSP 5000 series (GA Microcode Minimum: SVOS 7.3.1 83-05-02-
20/00)
Ensure Host Mode Options (HMO) 63 and HMO 114 set to ON.
See the blog at https://community.hitachivantara.com/s/article/Dealing-with-VMware-Datastore-space-management-
on-VSP-Storage-part-2 for additional details.
In vSphere 6.7 and later, VMware enabled additional performance parameters (low, medium, high, and fixed) to determine
the UNMAP throughput that arrays would receive. Setting automatic space reclamation settings to a fixed rate at 100
MB/secprovides a reasonable combination of UNMAP rate and storage processor (MPU) utilization. Always monitor MPU
usage before and after changing the UNMAP rate.
20
For VMFS 5, manual UNMAP is still supported with following command:
Storage DRS generates recommendations or performs Storage vMotion migrations to balance space use across the
datastore cluster. It also distributes I/O within the datastore cluster and helps alleviate high I/O load on certain datastores.
Trigger — This happens when space use on a datastore exceeds a threshold, or I/O latency on a datastore exceeds a
threshold.
Time — Storage DRS is invoked at the configured frequency which is, by default, every eight hours. It can also be invoked
when one or more datastores in a datastore cluster exceeds the user-configurable space utilization thresholds.
Granularity — VMware vCenter Server uses Storage vMotion to migrate virtual machine disks to other datastores in the
datastore cluster to balance resources.
When deciding which datastores to group into a datastore cluster, try to choose datastores that are as homogeneous as
possible in terms of the following:
The following are recommendations for VMware vSphere Storage DRS with Hitachi Storage:
Enable only Space metrics when a datastore cluster contains multiple datastores that are provisioned the same
dynamic provisioning pool with or without Hitachi Dynamic Tiering.
Moving a noisy neighbor within the same dynamic provisioning pool does not improve performance.
Enable Space and I/O metrics when a datastore cluster contains multiple datastores that are provisioned from different
dynamic provisioning pools.
Moving a noisy neighbor to the other dynamic provisioning pool balances out performance.
Granularity — Each virtual machine (or virtual disk) that accesses that datastore is allocated I/O resources in proportion
to its shares.
21
VMware vRealize Operations
VMware vRealize Operations helps visualize all the resources associated with a virtual machine in a single plane of glass. It
bridges the gaps between virtual and physical object to identify where the problem takes place. Hitachi Infrastructure
Management Pack for VMware vRealize Operations extends the functionality of vRealize Operations by providing simplified
management capabilities for Hitachi Storage components, improving performance and operational efficiency. It provides
better visibility into performance and storage capacity planning for your end-to-end virtual infrastructure environment.
Refer to Infrastructure Management Pack for VMware vRealize Operations for more information. Use the following set of
dashboards to manage capacity and performance of Hitachi Storage as shown.
FMD
SSD
SAS
SATA
External volumes
22
This helps improve the speed, capacity, and cost of performance. Dynamic Tiering improves underlying storage resources
with following conditions.
Trigger — Monitor the I/O load per page and relocate the page to the optimal tier
Time — Define a user-specified period of at least 30 minutes
Real-time with the active flash
Granularity — Relocate the storage tier with a page size of 42 MB
In addition, active flash monitors a page's access frequency level in real time to promote pages that suddenly became busy
from a slower media to high-performance flash media.
In a VMware environment, many workloads tend to be highly random with smaller block size. This may not be suitable for
deduplication and compression, even with an all flash configuration. Hitachi Dynamic Tiering with active flash may a good
option to improve capacity and cost by efficiently using the flash tier minimally.
The key factor affecting accommodation on a flash device is not performance, but capacity. So, this makes the high raw
capacity that the flash device has and the saving ratio that comes from deduplication and compression functionalities key
factors. See Data Reduction for more details.
Hitachi Storage Virtualization Operation System RF provides controller-based deduplication and/or compression
This capacity saving is processed at the dynamic pool or dynamic tiering level.
When you use FMD DC2 hardware-based compression, requires enabling the accelerated compression option on all
parity groups of FMD DC2 drives.
You can use either controller-based or hardware-based deduplication and compression, or a combination of both. With
a combination of both options, controller-based deduplication and hardware-based compression are used.
Use Hitachi Storage Virtualizing Operating System v9.x with controller-based compression for all systems, including NVMe
based systems.
23
When using FMD DC2, consider the following:
No performance degradation appears due to the truly hardware offloaded in-line or real-time accelerated
compression.
Regarding the compression saving ratio, the differences between software-based and hardware-based are
insignificant.
Inline processing-based compression provides you with reduction of initial capacity and cost.
Hitachi Storage Virtualization Operating System RF v9.x has changed how ADR is implemented to provide controller-based
deduplication by combining a mix of inline and post processing deduplication. Check the latest capabilities in Storage
Virtualization Operating System
From lab validation results at Hitachi, enabling deduplication achieved a 60-70% capacity saving for a datastore where
8 virtual machines with an operating system VMDK resides running Microsoft® Windows Server® 2012 R2.
Enabling FMD DC2 hardware accelerated compression enhances deduplication with more than a 20% capacity saving.
This combination of deduplication and compression achieved more than 80-90% capacity savings in total.
A main concern related to deduplication is performance degradation. This comes from mainly the following two factors:
It consumes extra storage compute resources to perform deduplication and metadata management.
The garbage collection running as a background task also requires processing overhead. This task may increase
storage CPU (MP) usage from 2% to 15%.
The following are some of the considerations with regards to controller-based deduplication:
It may impact I/O performance. Verify the performance by utilizing best practices or the cache optimization tool (COT)
tool before using the capacity saving function.
Because approximately 10% of the capacity is used for metadata and garbage data, the capacity saving function should
be applied only when the saving is expected to be 20% or higher.
In deduplication and compression, processing is performed per 8 KB. Therefore, if the block size of the file system is an
integral multiple of 8 KB, then the capacity saving is likely to be effective.
The capacity saving function is not a good fit for high-write workloads. If the write workload rate is higher than garbage
collection throughput, then the storage cache write-pending increases, causing performance degradation.
The capacity saving effect vary depends on your application and workload. You need to know your application workload and
suitability before enabling a capacity saving feature. Table 3, “Deduplication Consideration for General Use Cases,” on
page 23 lists the possible use cases for capacity savings.
24
TABLE 3. DEDUPLICATION CONSIDERATION FOR GENERAL USE CASES
Database (TPC-H) Deduplication is not effective because the database has unique information for each block.
Database (TPC-C) For a database that has many data updates, garbage data is increased, so it is not suitable.
Image/video Compressed by application.
Backup/archive Deduplication is effective between backups.
If your application requires high IOPS and low latency, and if your data is compressible, FMD DC2 accelerated
compression (without dedupe) might be an option.
RAID-6 is the recommended RAID level for pool-VOLs, especially for a pool where recovery time from a pool failure due
to a drive failure is not acceptable.
Configure a parity group across the drive-boxes to maximize the performance by increasing the number of back-end
paths.
Standard Storage SRM and Stretched Storage SRM with Global-active Device Best
Practices
VMware vCenter Site Recovery Manager integrates tightly with Hitachi Storage arrays using either Hitachi Storage
Replication Adapter or Hitachi Ops Center Protector. This provides centralized management of recovery plans. Tight
integration between storage systems, VMware vCenter, VMware vCenter Site Recovery Manager, and Hitachi storage
replication adapters ensures a coordinated recovery for large, business critical environments.
Remote data replication is a key function in building out stable and reliable disaster recovery environments. Replicating
data to a remote secondary site represents the most effective insurance policy against a catastrophic failure. Although you
can perform data replication at the server level, you can perform data replication more effectively within the storage
infrastructure.
25
The following are two types of underlying storage configurations supported by VMware Site Recovery Manager:
Standard storage (active-standby solution) leveraging Hitachi True Copy or Hitachi Universal Replicator
Stretched storage (active-active solution) leveraging global-active device on Hitachi Virtual Storage Platform
Table 4 lists the differences between two types of storage.
Storage Site failover is required due to When primary storage failure occurs, no site failover is required by
availability primary storage failure. using the cross path to remote site storage which is virtualized as a
single stretched storage and volume across the sites powered by
It costs application downtime. global-active device technology.
Simplicity Simple because traditional In addition to traditional disaster recovery configuration, there is a
disaster recovery configuration need to consider quorum storage and additional paths between sites as
consists of primary storage and cross-paths, and so forth. It tends to be more complex and larger.
secondary storage.
You may decide, depending on required RPO, which results in which replication type you choose.
Hitachi Block Storage Replication Adapter for VMware vCenter Site Recovery Manager (SRM) – Deployment Guide
Hitachi Data Instance Director Adapter for VMware Site Recovery Manager (SRA)
26
Global-Active Device with Native Multi-Pathing with ALUA
For a VMware Metro Storage Cluster configuration, global-active device with VMware native multi-pathing (NMP) with
ALUA is supported with micro code 83-03-01-x0/00 and later. This feature allows you to present I/O to the remote site
storage across long distances path that cause high response time by specifying it as non-optimized path. This minimizes
response time and the cost of WAN traffic. It is recommended to turn on this feature with site distances greater than 20
miles (32 km).
Here is an example of enabling ALUA mode and specifying non-optimized path on Hitachi Storage:
Here is an example of enabling a SATP rule set as ALUA for Hitachi devices and selecting PSP as round robin on the VMware
ESXi side:
Esxcli storage nmp satp rule add -V HITACHI -M “OPEN-V” -P VMW_PSP_RR -s VMW_SATP_ALUA -c tpgs_on
Hitachi recommends using RR (round robin) instead of MRU (most recently used).
With VMware vSphere 6.7U1 and vSphere 6.5 P03 or later, this ALUA SATP rule is set by default. See VMware Native
Multipathing rules for Hitachi VSP now enabled by default in vSphere builds for more details.
Uniform host access configuration — This is when VMware ESXi hosts from both sites are all connected to a storage
node in the storage cluster across all sites. Paths presented to ESXi hosts are stretched across this distance.
Non-uniform host access configuration — This is when VMware ESXi hosts in each site are connected only to storage
node or nodes in the same site. Paths presented to ESXi hosts from storage nodes are limited to the local site.
Refer to Implement vSphere Metro Storage Cluster with Hitachi Virtual Storage Platform (VSP) (2145375) for more
information regarding to this topic.
3 Data Center (3DC) with VMware Site Recovery Manager Best Practices
The 3DC solution consists of clustered primary and secondary datacenters leveraging global-active device in Hitachi
Virtual Storage Platform, and the third data center which is replicated from the others as a disaster recovery site leveraging
Hitachi Universal Replicator with delta resync functionality.
Hitachi Universal Replicator with delta resync functionality establishes storage remote replication from the primary data
center to the third data center and from the secondary data center to the third datacenter, respectively. This is called the
global-active device 3DC delta resync environment, as shown.
27
To maintain adequate service level agreements, ensure journals are adequately sized to avoid any throttling of IOPS to
maintain replication SLAs if Hitachi Universal Replicator inflow control is set to enabled. If Universal Replicator inflow
control is set to disabled, host I/O is prioritized and the Universal Replicator relationship is suspended.
Installing VMware Site Recovery Manager in this 3DC environment gives you the orchestrated and repeatable planned or
unplanned migration or disaster recovery operations using a tested and proven recovery plan. This enables end-to-end
virtual machine protection across 3 datacenters. As a normal state, VMware SRM protects the virtual machine between the
primary and the third data center.
This solution is based on VMware Metro Storage Cluster, which clusters the primary and the secondary data centers within
a single VMware vCenter data center object and uses stretched storage cluster powered by global-active device.
When the primary datacenter goes down, the virtual machine can be restarted on the secondary data center, leveraging
VMware vSphere High Availability fail over functionality as a VMware Metro Storage configuration. During failover from the
primary to secondary datacenter, storage remote replication established from the primary to the third data center is also
automatically failed over to the other one established from the secondary to the third data center by leveraging delta resync
functionality.
For a global-active device 3DC delta resync environment solution, the virtual machine protected by VMware SRM can follow
this datacenter failover movement and re-protect the virtual machine between the secondary and third datacenter with
minimal effort.
28
Note — As a normal state, VMware SRM protects the virtual machine between the primary and third datacenter. When
the primary datacenter goes down, storage remote replication can automatically failover though, the re-protection of
virtual machines by SRM requires some manual operation to switch the source and target datacenter. To do this,
switching the command control interface configuration file and restarting the Hitachi Open Remote Control Manager
daemon is required.
VMware Cloud Foundation (VCF) supports Hitachi SAN storage (VMFS and vVols) used as principal or supplemental
storage. The use cases vary by customer, ranging from flexibility to scale storage footprint, mission critical applications
with stringent RPO/RTO requirements, to simply matching business outcomes to suitable tier of storage.
VCF 4.x+ expanded their external storage offerings for principal or supplemental storage with support for vVols and VMFS
to augment native vSAN datastores. The best practice is to deploy the VASA provider OVA from Hitachi in the VCF
management domain and each VCF workload domain VMware vCenter can register to that single VASA provider.
29
30
Kubernetes and Persistent storage Options with Hitachi VSP and UCP
Review the recently published multi-cloud container reference architecture which walks you through how to deliver and
manage a VMware Tanzu-based container workload platforms as part of your evolving infrastructure. This paper
demonstrates combining VMware VCF, TKGs, TMC, and vVols on VSP, vSAN, Velero, Hitachi UCP, and HCP to
manage/protect on-premises Kubernetes clusters and workloads including persistent storage workloads. Previous to
this, there was a good paper which covered Deployment Options for Kubernetes Container Applications on Unified
Compute Platform CI with Hitachi VSP Series Reference Architecture Guide (PDF) for using persistent cloud native
storage (CNS) for containers (vSAN, vVols, VMFS, and bare metal).
With the growth in containers, there will be the need for persistent storage for PostgreSQL databases as targets for those
container services. These services will access their ‘persistent data’ through shared but probably sharded databases
instances. Hitachi storage can provide persistent storage for these container services.
Additionally, you can take advantage of the additional work done to provide storage policy management for persistent
storage for Kubernetes clusters, including Tanzu Kubernetes guest clusters, running on top of VMware vSphere to deliver
the requested storage class capabilities at the vmdk level.
Hitachi Virtual Storage Platform G series storage provides a hybrid media powered platform. It has a broad range of
efficiency technologies that deliver maximum value while making ongoing costs more predictable. You can focus on
strategic projects and consolidate more workloads while using a wide range of media choices.
Hitachi Storage Virtualization Operating System RF is at the heart of the Virtual Storage Platform F and VSP G series family.
It provides storage virtualization, high availability, flash optimized performance, quality of service controls, and advanced
data protection. This proven, mature software provides common features, management, and interoperability across the
Hitachi portfolio. This means you can reduce migration efforts, consolidate assets, reclaim space, and extend product life.
31
Hitachi Unified Compute Platform
Hitachi Unified Compute Platform (UCP) is a converged system that provides the foundation for business transformation
and IT modernization. It allows you to build a modern IT infrastructure than can host any application, at any scale, and at any
location.
The Unified Compute Platform architecture consists of modular building blocks including VMware vSAN and external
storage-based converged systems. Its components scale independently to provide you with greater configurability,
flexibility, and agility. The solution may be configured to support traditional “systems of record” as well as cloud-native
applications.
it provides operational workflows for end-to-end provisioning of VMware ESXi hosts, bare metal hosts, datastores, and
networking orchestration.
32
For More Information
Hitachi Vantara Global Services offers experienced storage consultants, proven methodologies and a comprehensive
services portfolio to assist you in implementing Hitachi products and solutions in your environment. For more information,
see the Services website.
Demonstrations and other resources are available for many Hitachi products. To schedule a live demonstration, contact a
sales representative or partner. To view on-line informational resources, see the Resources website.
Hitachi Academy is your education destination to acquire valuable knowledge and skills on Hitachi products and solutions.
Our Hitachi Certified Professional program establishes your credibility and increases your value in the IT marketplace. For
more information, see the Hitachi Vantara Training and Certification website.
For more information about Hitachi products and services, contact your sales representative, partner, or visit the Hitachi
Vantara website.
Hitachi Vantara
Corporate Headquarters Regional Contact Information
2535 Augustine Drive USA: 1-800-446-0744
Santa Clara, CA 95054 USA Global: 1-858-547-4526
www.HitachiVantara.com | community.HitachiVantara.com HitachiVantara.com/contact
© Hitachi Vantara LLC, 2020. All rights reserved. HITACHI is a trademark or registered trademark of Hitachi, Ltd. VSP is a trademark or registered trademark of Hitachi Vantara
LLC. Microsoft, Windows Server, and Microsoft Office are trademarks or registered trademarks of Microsoft Corporation. All other trademarks, service marks, and company
names are properties of their respective owners.
Notice: This document is for informational purposes only, and does not set forth any warranty, expressed or implied, concerning any equipment or service offered or to be offered
by Hitachi Vantara Corporation.
MK-SL-145-02, May 2020