A Buyer's Guide

to Enterprise
Kubernetes
Solutions
A Comparison of Pivotal PKS,
Platform9 Managed Kubernetes,
Rancher, and Red Hat OpenShift
In just a few years, Kubernetes has rapidly emerged as the de-facto
open source standard for container orchestration. Numerous Kubernetes
solutions and products have emerged from startups to established
traditional vendors thus making it difficult to compare and contrast the
various offerings. This guide identifies 18 technical and operational
capabilities to consider while evaluating various vendor offerings and
then provides a detailed comparison of the level of completeness of
these capabilities for four leading solutions in this market: Pivotal PKS,
Red Hat OpenShift, Rancher, and Platform9 Managed Kubernetes.

What are the key features to consider while evaluating an enterprise

Kubernetes platform?

1. Provisioning of Kubernetes Clusters, High • Hosted Kubernetes as a service (KaaS) - A vendor

Availability and Healing will offer Kubernetes as a service on top of
Kubernetes does not offer deployment of highly infrastructure that’s hosted by the provider
available clusters out of box and must be configured
by the Kubernetes administrator. It is recommended • Hybrid Cloud Kubernetes as a Service
that at least three master nodes are configured - Kubernetes is offered as a service on the
behind a load balancing solution with integrated infrastructure of your choice, your own
or independent clustered deployment of etcd that infrastructure or public cloud
stores all the cluster state information. Any high
availability solution must also account for failure 3. Prerequisites and Operating System Requirements
scenarios and auto-repair and recovery. The prerequisites of an enterprise Kubernetes
solution define what infrastructure requirements you
2. Deployment Model(s) Supported need to satisfy before you can get up and running
The deployment model of a Kubernetes solution with Kubernetes. Some solutions require an expensive
defines how it will integrate within your enterprise licensing purchase of underlying infrastructure,
environment and what level of support service such as a hypervisor, or an investment in a hosted
level agreement (SLA) it can provide for day 2 Kubernetes solution.
operations. The top three deployment models for
Kubernetes solutions are: 4. Monitoring and Operations Management
A production Kubernetes cluster must be monitored
• Traditional on-premises deployment - users at all times to handle any issues and outages
download and deploy Kubernetes on their without severely affecting cluster and application
infrastructure on their own or using professional availability to users. An enterprise Kubernetes
services and support from a vendor solution must provide this capability out of box.

2
5. Cluster Upgrades strategy by investing in multiple cloud solutions.
Kubernetes has a large community of contributors Having multiple private and/or public clouds as part
and a new version is available every 3 months. An of your cloud strategy ensures that you aren’t getting
enterprise-class solution will support rolling upgrades locked into a single provider with no leverage on pricing,
of clusters, such that the cluster and the cluster API to have high availability across your infrastructure
is always available even while the cluster is being overall, and to satisfy your unique business policies.
upgraded. Additionally, it will provide the ability to
11. Enterprise Grade User Experience
rollback to previous stable version upon failure.
Enterprise grade user experience is all about having
6. Multi-cluster Management a polished user interface that enables enterprises to
A single Kubernetes cluster can scale horizontally to manage their hybrid environments though a single
support large sets of workloads. However, running UI. This delivers complete visibility simplifying
Kubernetes in production requires being able to run communications across the environment. This UI
multiple Kubernetes clusters, as you will want to fully isolate should allow operations that span multiple clusters:
your dev/test/staging applications from production for example, globally searching for workloads of a
applications by deploying them on a separate cluster. specific type or tagged with a specific label across
all clusters running on different regions, data
7. Multi-tenancy, Role-based Access Control and centers and cloud providers.
Single Sign-on Support
Kubernetes supports multi-tenancy at the cluster 12. Application Lifecycle Management -
level using the namespace abstraction. However, in Application Catalog
a multi-cluster environment, you need a higher level Application catalog provides easy one-click deployment
multi-tenancy abstraction to supplement Kubernetes for a set of pre-packaged applications on top of
multi-tenancy and provide the right level of isolation Kubernetes. It also provides end users a vehicle to build
across different teams of users. It should integrate and publish their own applications via the catalog for others
with Single-Sign On (SSO) solutions most commonly in their team or their organization to deploy in a one click
used by enterprises such as Active Directory or manner. The application catalog enables organizations to
ADFS, Okta, and other popular SAML providers. standardize on a set of application deployment recipes
or blueprints, avoiding sprawl of configurations.
8. Load Balancing
Kubernetes automatically load balances requests to 13. Production Grade Service Level Agreements (SLA)
application services inside of a Kubernetes cluster. As more and more organizations are running their business
However, some services need to be exposed externally on Kubernetes, IT must ensure that it can support the SLAs
for consumption by outside clients. Kubernetes does that the business requires. IT must ensure that Kubernetes
not provide an out-of-the box load balancing solution is available to developers and the business to support key
for that type of services. An enterprise Kubernetes initiatives. Most organizations require 99.9% uptime.
product should include a robust external load balancing
14. Ease of Setup, Installation, Continuous
solution, or integrate seamlessly with existing commercial
Use, Management, and Maintenance
load balancers.
A successful Kubernetes platform must be easy to
9. Private Registry Support and implement and maintain so organizations can leverage
Image Management containers continuously. This alone is a major barrier
Running containerized applications on Kubernetes clusters that many organizations do not overcome.
requires having access to a container registry where your
15. Networking Support and Integrations
application images will be stored. A large enterprise
Networking integration is a critical component of
organization will typically want a secure private container
running Kubernetes clusters in production and at scale.
registry to store their proprietary application images. An
An enterprise will typically want Kubernetes to integrate
enterprise Kubernetes solution should provide image
with a Software-Defined-Networking (SDN) solution
management capability out of box.
of their choice that they currently standardize on or a
10. Hybrid Cloud Integrations and APIs container native solution such as calico or weave that
Every enterprise today wants to build a cloud neutral gives them more options around isolation.

3
16. Storage Support and Integrations
Similar to networking, integration with enterprise
grade storage is an essential component of running
Kubernetes clusters in production. Kubernetes provides
an abstraction called Persistent Volumes to hold data
persisted by stateful applications. It is important for a
Enterprise Kubernetes product to map PVs to an actual
highly-available storage technology. Enterprises will
typically want their Kubernetes deployment to integrate
with storage solutions that they have already deployed
such as NetApp, Pure, SolidFire, etc. or they may want
to integrate with a container native storage technology
such as Portworx.

17. Self Service Provisioning

Developers must have self-service access to one or
more Kubernetes clusters with right levels of isolation
in place so only members with right privileges can
access production workloads.

18. Built-inCI/CD Support

One of the most critical workloads run by the developers
is Continuous Integration / Continuous Delivery. A robust
CI / CD pipeline is critical to ensure agile development
and rapid delivery of new software releases to customers.

4
Comparison Scorecard
PLATFORM9
RED HAT
FEATURES MANAGED RANCHER PIVOTAL PKS
OPEN SHIFT
KUBERNETES

Provisioning of Kubernetes Clusters

High Availability and Healing

Deployment Model(s) Supported

Prerequisites and Operating System Requirements

Monitoring and Operations Management

Cluster Upgrades

Multi-cluster Management

Multi-tenancy, Role-based Access Control, and

Single Sign-on Support

Load Balancing

Private Registry Support and Image Management

Hybrid Cloud Integrations and APIs

Enterprise Grade User Experience

Application Lifecycle Management -

Application Catalog

Production Grade Service Level Agreement

Ease of Setup, Installation, Continuous Use,

Management, and Maintenance

Networking Support and Integrations

Storage Support and Integrations

Self Service Provisioning

Built-in CI/CD Support

5
Detailed Comparison Table
PLATFORM9 MANAGED RED HAT
PRODUCT RANCHER PIVOTAL PKS
KUBERNETES OPEN SHIFT

Provisioning of Fully automated provi- Fully automated Fully automated Fully automated
Kubernetes Clusters sioning of clusters provisioning of clusters provisioning of clusters provisioning of clusters

High Availability »» Built-in support »» Leverages native »» Supports a highly »» Includes Cloud
and Healing for highly available Kubernetes features to available cluster Foundry Ops
clusters out of the box deliver HA and healing deployment Manager Provides
BOSH which is a vital
»» Clusters of 1/3/5 »» The default HAProxy component within PKS
masters are supported load balancer can
for quorum be used to create »» BOSH monitors the
a multi-master and health of clusters and
»» Built-in etcd high multi-etcd cluster enables self-healing
availability support
environment - with etcd to optimize capacity
»» Supports full repair or nodes either forming Unhealthy nodes
recovery of etcd upon their own cluster or are automatically
failure deployed on the same detected and
node as the master resurrected without
downtime

Deployment Model(s) One deployment »» Traditional deployment Three deployment One deployment model
Supported model offered: and support model models offered: offered across three
with software platforms. The product
»» SaaS-managed »» Hosted Public
downloaded files are downloaded for
offering - ideally suited Cloud Offering - trial
and installed Pivotal Ops Manager
for enterprise hybrid environment only
and PKS and the end-
clouds, includes sup- »» Support for on-prem »» OpenShift Dedicated user performs the install.
port, upgrades, remote and public cloud
- Single-tenant, VMware vSphere, AWS,
management, etc. deployments
high-availability and GCP supported
OpenShift clusters
hosted on Amazon
Web Services Delivered
as a hosted service

»» OpenShift Container
Platform - Enterprise
offering available for
large customers with
on-prem and/or
hybrid infrastructure

Prerequisites and Supports all popular Supports all popular »» Supports Red Hat PKS has an extensive
Operating System enterprise Linux distribu- enterprise Linux distribu- Linux only list of prerequisites:
Requirements tions - Red Hat, CentOS, tions - Red Hat, CentOS,
»» RHEL subscription »» It can not be installed
Ubuntu Ubuntu
is required and on any linux systems
included as part of the
»» Requires VMware
OpenShift bundled
vSphere Enterprise
product subscription
Plus Edition or
vSphere with
Operations
Management
Enterprise Plus
minimum

6
 PLATFORM9 MANAGED RED HAT
PRODUCT RANCHER PIVOTAL PKS
KUBERNETES OPEN SHIFT

Monitoring and »» 24 x 7 live monitoring »» Performs health »» Diagnostic tools via »» Does not provide any
checks on all command line for out of the box live
Operations »» 99.9% guaranteed SLA Kubernetes clusters, health statistics monitoring for your k8s
Management »» Proactive repair and presents resource clusters, Prometheus
»» Environmental health
»» Automated email consumption statistics
check information
support, etc.
notifications for any »» Sends cluster-level »» Prometheus available »» Traditional support
issues alerts for Kubernetes ticketing system
but requires lot of
»» Automated support system components
manual configuration
ticket creation and (e.g., etcd, DNS,
for storage, alerting
triaging of issues etc.) -Customizable
etc
resource alerts such
as CPU, memory etc.

»» Traditional support
ticketing process for
issues

»» Automatically deploys
and configures
Prometheus for
monitoring

Cluster Upgrades »» Fully automated »» Providers an easy built Administrators need to »» Patching and upgrades
cluster upgrades in cluster upgrade manually initiate upgrades of Kubernetes nodes
delivered seamlessly, experience to clusters upon installing can be managed from
with no interruption new version of OpenShift the PKS platform in
»» Upgrade one cluster
to the environment a centralized fashion,
at a time or multiple
without impact to
»» Zero-downtime clusters simultaneously
running applications
upgrade
via BOSH

»» Customers are
guaranteed to always
have the latest
Kubernetes version
available to them

»» PKS maintainst
compatibility to
Google Container
Engine (GKE)

Multi-cluster »» Built in multi-cluster »»Provides unified »» A typical deployment »» Supports multi-cluster

Management support. Create any management of creates a single management. Create
number of clusters multiple clusters Kubernetes cluster that one or more clusters
is designed to scale up using PKS CLI, then
»» Admins can manage »» Build clusters on to 2000 nodes list the clusters, view
multiple clusters across public cloud providers
cluster details and
different regions, data like GKE, EKS, AKS or »» All users of that delete clusters using
centers and clouds on data centers deployment are
the CLI
expected to share
»» Able to discover that single cluster
existing clusters
and achieve isolation
pre-created
via a combination of
Kubernetes name-
spaces, and OpenShift
multi-tenancy

»» Multiple clusters
achieved through
multiple OpenShift
deployments

7
 PLATFORM9 MANAGED RED HAT
PRODUCT RANCHER PIVOTAL PKS
KUBERNETES OPEN SHIFT

Multi-tenancy, »» Support for multi-re- »» Provides centralized »» Delivers multi-tenancy »» Supports multi-ten-
Role-based Access gion management. authentication through projects, ancy in a limited
Control, and Single Built in multi-tenancy (GitHub, AD/ called Kubernetes form
Sign-on Support support LDAP, SAML, etc.) namespaces
»» On VMware vSphere
across RKE or cloud
»» Kubernetes RBAC is »» Kubernetes RBAC with NSX-T, PKS
Kubernetes services
fully supported is utilized to define uses VMware NSX-T

»» Full support for »» Allows admins to granular access to isolate different

define Kubernetes policies for users clusters using NSX
Single-Sign On (SSO).
RBAC policies and security policies
Integrate with a »» There is no cross
network and pod
SAML-based provider cluster multi-tenancy »» Kubernetes clusters
security policies cen-
that your organization can be deployed into
trally and apply them
uses such as Okta, different vSphere
across any cluster
ADFS, Ping Identity, clusters and config-
etc. ured to use different
datastores

»» Multi-tenancy is
not available when
deployed without
NSX-T or on public
clouds

Load Balancing »» Out of the box »» Leverages native »» Out of the box support »» When deployed on
support for cluster Kubernetes features to for cluster and service vSphere with NSX-T,
and service level deliver load balancing level load balancing PKS can leverage load
load balancing with with default HAProxy balancing feature
MetalLB load balancer load balancer from NSX-T to provide
cluster and service
»» Can integrate with level load balancing
customer specific load
balancers(AVI and »» When deployed on
others) public clouds like
GCP or AWS, PKS
leverages the cloud
native load balancing
capability

Private Registry »» Does not provide out »»Does not provide The internal integrated »» IncludesProject
Support and Image of the box support for out of the box Docker registry can Harbor which is
Management private registries support for private be deployed in the an open source
registry deployment. OpenShift environment enterprise container
»» Registries and Users have to deploy to locally manage images. registry. Harbor
secrets required to
their own public or OpenShift does not han- simplifies image
authenticate with the
private registries dle DockerHub well with management with
registries need to
their private registries. distribution,
be managed by the »» Does support storing Because of complex replication and security
customer separately of secrets required
security requirements and mechanisms
to pull images from
configs specific to open-
private registries »» The Harbor registry
shift, a user is prevented
uses a logical construct
from pulling a docker
called Project
image. A user would have
to build image streams, »» This is used to group
secrets management, users and repositories
and built to image in a to enable fine-grained
lot of cases, unlike pure access control
kubernetes which is much
less complex.

8
 PLATFORM9 MANAGED RED HAT
PRODUCT RANCHER PIVOTAL PKS
KUBERNETES OPEN SHIFT

Hybrid Cloud »» Includes the most »» Automates cluster »» Provides a managed »» Supports on-premises
Integrations and APIs native integration with creation on bare metal deployment on AWS and public cloud
all major private data servers, VMware, or based deployments
»» A joint collaboration
center/private cloud any IaaS cloud
has been announced »» On-premises
offerings and major
»» Clusters on IaaS with Microsoft to requires vSphere or
public cloud providers
clouds are created support OpenShift vSphere with NSX-T
»» Integrates natively with cloud specific deployment on Azure
»» Does not support
with VMware version of Kubernetes
pure bare metal
vSphere, Linux/KVM, (EKS for Amazon AWS
deployment, or
OpenStack cloud, AKS for Azure
Linux/KVM virtualized
cloud, etc.)
»» Clusters on public environment
clouds are created »» Rancher is subject
with the public cloud’s to any limitations
IaaS layer to provide the cloud providers
a native Kubernetes expose around
cluster experience versioning, update,
HA, etc.

Enterprise Grade User »» Provides and enterprise »» Includes an intuitive UI »» Provides a native »» Lags behind signifi-
Experience class UI and user that makes it easy for UI that enables cantly in this category
experience users to deploy services management of your
»» There is no UI for
on Kubernetes and get Kubernetes resources
»» The clarity UI provides PKS. All Kubernetes
complete visibility and the catalog
a single pane of glass operations such as
across bare metal, »» Common configuration »» This is a web console cluster creation and
virtualized and options directly from the only and does allow for management happen
containerized UI for defining scheduling customization purely via CLI
workloads rules, health checks,
»» PKS does enable
ingress controllers,
»» Unify all your data the Kubernetes
secrets, storage and
centers, private dashboard
other key configuration
clouds, and public
choices are offered
clouds under single UI

Application Lifecycle »» Built in support for »» Built in support for »» An extensive »» Does not ship with a
Management - Application catalog Application catalog application catalog built-in Kubernetes
Application Catalog that’s populated with that’s populated with and PaaS layer helps application catalog
public Helm chart public Helm chart with building and
»» It is able to deploy
applications applications deploying apps
Helm charts
»» Administrators can »» ‘Rancher certified’ »» The service catalog is
provide users access applications provided based on Open Service
to applications that in the catalog that are Broker API
are private to the tested and certified
»» It ships with two service
organization by Rancher
brokers, one to enables
»» Support for applications from their
managed apps built in app template
library, the other is an
ansible broker

»» The templated appli-

cations support - Rails
(Ruby), Django (Python),
Node.js, CakePHP
(PHP), and Dancer (Perl)

»» The Ansible broker

supports integration
w/ Ansible Playbook
Bundles (APB)

»» The service catalog

offers Prometheus, EFK,
Jenkins etc.

9
 PLATFORM9 MANAGED RED HAT
PRODUCT RANCHER PIVOTAL PKS
KUBERNETES OPEN SHIFT

Production Grade »» Platform9 contractu- »» Provides a traditional »» Provides a traditional »» Provides a traditional
Service Level ally promises 99.9% enterprise class enterprise class enterprise class
Agreement cluster uptime and support model support model support model
high availability
»» Troubleshooting »» Troubleshooting »» Troubleshooting
»» Provides self healing, is handled via is handled via is handled via
problem resolution support tickets support tickets support tickets
through the service
»» Customers drive the »» Customers drive the »» Customers drive the
manual upgrades and any manual upgrades and any manual upgrades and
issues require support issues require support any issues require support
team engagement team engagement team engagement

Ease of Setup, »» Platform9’s SaaS »» Simple setup, run »» Installing and config- »» The installation of PKS
Installation, managed gets a single docker uring OpenShift is a is manual and requires
Continuous Use, Kubernetes up and command on a linux manual process which Pivotal Ops Managed
Management, and running in minutes machine and you are is ansible-based Several to be configured for
Maintenance up and running ansible playbooks are on-prem and public
»» Create a simple required during the cloud use
Kubernetes cluster »» Provides an intuitive UI installation
using on-prem servers, to help with rest of the »» Any problems require
VMs or public cloud setup and Kubernetes logging a support
resources in minutes cluster creation ticket

»» Manage clusters with

one-click UI-based
upgrades and
troubleshooting

Networking Support »» Provides full CNI »» Rancher provides CNI »» OpenShift provides »» Supports integration
and Integrations support support CNI support and can with VMware NSX-T for
integrate with any CNI advance networking
»» Integrates OOB with »» Out of the box based SDN on VMware.
Flannel, Calico, Weave support provided for
and OpenContrail canal, calico, flannel »» By default OpenShift »» It also support
SDN is deployed, which flannel, calico,
»» Other CNI compatible configures an overlay nuage, OVN and
integrations possible
network using Open kube-router
on customer request
vSwitch (OVS) and
supports 3 modes:
1. Flat network model
with ovs-subnet
plugin where every
pod can communicate
with every other pod

2. Project level isolation

for pods

3. Services using
ovs-multitenant
plugin, and which
enables administrators
to configure their own
isolation policies using
Network Policy objects
with ovs-network
policy plug-in

»» Out of the box third

party CNI plugins
supported: Flannel,
Nuage and Kuryer

10
 PLATFORM9 MANAGED RED HAT
PRODUCT RANCHER PIVOTAL PKS
KUBERNETES OPEN SHIFT

Storage Support and »» Supports integration »» Rancher supports »» Supports integration »» Requires k8s vSphere
Integrations with any flexvolume flexvolume driver with network based cloud provider so that
drivers level integration with persistent storage vSphere persistent
storage providers using the Kubernetes storage and data
»» Integrates with any persistent volume services are exposed at
cinder supported storage
framework container volume level
backend. - Supports
integration with all »» Supports a wide »» Cluster admins can
popular storage variety of persistent create storage classes
backends such as storage endpoints that can map to a
NetApp, Pure Storage, etc. such as NFS, specific underlying
GlusterFS, OpenStack vSphere datastore or
Cinder, FlexVolume, a VSAN datastore with
VMware vSphere etc desired configuration,
or by referencing
a vSphere storage
policy name

»» For deployment on
public clouds, PKS
integrates with cloud
specific persistent
storage offerings

Self Service »» Complete self-service Provides a complete Provides a self-service »» Does not include a UI
Provisioning provisioning enabled self-service provisioning UI that is separate from for Kubernetes, which
via Platform9’s clarity UI UI for end users and the default Kubernetes means there is no self
admins to create dashboard UI to enable service capabilities
»» Users log into the UI workloads on top of self-service for develop- for Administrators
as part of a specific
Kubernetes ers and administrators to manage their k8s
Tenant (eg dev/test/
clusters
production) and are
able to access clusters »» End-users cannot log in
provided they have and deploy or manage
been granted access their workloads

»» Quick deployment of »» The Kubernetes

pods, deployments and dashboard UI can be
services via a wizard used out-of-the box
for cluster creation

11
 PLATFORM9 MANAGED RED HAT
PRODUCT RANCHER PIVOTAL PKS
KUBERNETES OPEN SHIFT

Built-in CI/CD Support »» Provides Spinnaker and »» Includes integrated CI/ »»Pipelines and Build Does not ship with a CI/
Jenkins via the Helm CD, making it easy for Strategies simplifies CD tool by default
application catalog teams using Kubernetes, the creation and
to quickly integrate it automation of
with their development, dev/test and
testing and release production pipelines
management process
»» Ships out of the box
»» Users can easily with a Jenkins build
point Rancher at any strategy and client
git repo and it will plugin to create a
automatically run builds Jenkins pipeline.
on Kubernetes, deploy However, the setup to
test environments, and create and configure
move product production pipelines
is manual and time
consuming.

»» The pipeline build

configuration creates
a Jenkins master pod
(if one doesn’t exist)
and then automatically
creates slave pods to
scale jobs & assign
different pods for jobs
with different runtimes

12
Platform9 Managed Kubernetes
Platform9 Managed Kubernetes is the industry’s only SaaS-based
continuously managed Kubernetes service that provides:

»» Guaranteed 99.9% uptime SLA

»» Remote Monitoring and Healing
»» Remote Upgrading and Instant Security Patching
»» Central management across on-prem data centers, public clouds, and at the Edge.
»» Upstream open source Kubernetes

To learn more, please visit

platform9.com/managed-kubernetes

About Platform9
Platform9 (platform9.com) delivers a SaaS-managed hybrid cloud solution that turns existing infrastructure into a cloud, instantly. We help enterprises drive digital transformation by enabling them
to manage VMs, Containers and Serverless Functions on ANY infrastructure — on-premises, in public clouds, or at the edge – with a self-service, simple and unified experience. Customers such
as Cadence, Autodesk, Veritas, Nanometrics, EBSCO, Bitly, LogMeIn, and Aruba see upwards of 300 percent improvement in IT efficiency, 33 percent faster time to market, and 50-80 percent
improvement in data center utilization and cost reduction. The company is headquartered in Sunnyvale, CA, and is backed by Redpoint Ventures, Menlo Ventures, Canvas Ventures, and HPE.