Kubernetes on AWS

Pre-reqs:
● Git: https://git-scm.com/book/en/v2/Getting-Started-Installing-Git
● AWS CLI: http://docs.aws.amazon.com/cli/latest/userguide/installing.html
● kubectl:
http://cs-k8s-workshop.s3.amazonaws.com/kubectl/darwin/amd64/kubectl
http://cs-k8s-workshop.s3.amazonaws.com/kubectl/linux/amd64/kubectl
● Bash
● git clone https://github.com/ContainerSolutions/kubernetes-aws-workshop

www.container-solutions.com | info@container-solutions.com
Kubernetes on AWS
Grant Ellis
grant.ellis@container-solutions.com

Who’s who
● Presenters
● You!
➔ Developers? Ops? DevOps?
➔ Tools, languages & frameworks?
➔ Familiar or using any orchestration platform? Mesos/Swarm/ECS?

Purpose of the Workshop
● Get an overview of the components in kubernetes
● See how kubernetes leverages features present in AWS
● Get an idea of how a production setup may take shape

Scope of the Workshop
● Basic features of Kubernetes
● Brief look at AWS CloudFormation and IaaS components
● Hands on

Kubernetes

Kubernetes
● From the Greek meaning “Helmsman” or “Pilot”
● Founded by Joe Beda, Brendan Burns and Craig McLuckie
● First announced by Google in 2014

Basic concepts
● Pods
● Labels / Selectors
● Replication Controllers / Replica Sets
● Deployments
● Services
All Resources can be expressed as YAML or JSON files

Pods
● A pod is one or more containers
● Ensures co-location / shared fate
● Pods are scheduled, then do not move between nodes
● Containers share resources within the pod:
➔ Volumes
➔ Network / IP
➔ Port space
➔ CPU / Memory allocations

Pod example
apiVersion: v1
kind: Pod
metadata:
labels:
name: influxdb
name: influxdb
spec:
containers:
- image: docker.io/tutum/influxdb:latest
name: influxdb
ports:
- containerPort: 8083
name: admin
protocol: TCP
- containerPort: 8086
name: http
protocol: TCP

Labels / Selectors
● Labels are arbitrary metadata
● Attachable to nearly all API objects
➔ e.g.: Pods, ReplicationControllers, Services...
● Simple key=value pairs
● Can be queried with selectors

Labels example
- release=stable, release=canary
- environment=dev, environment=qa, environment=prod
- tier=frontend, tier=backend, tier=middleware
- partition=customerA, partition=customerB
- etc…

Labels example

Selectors explained
Labels are queryable metadata - selectors can do the queries:
- Equality based:
- environment = production
- tier != frontend
- combinations: tier != frontend, version = 1.0.0
- Set based:
- environment in (production, pre-production)
- tier notin (frontend, backend)
- partition or !partition

Selectors example

Replication Controllers
● Define the number of replicas of a pod
● Will scheduled across all applicable nodes
● Can change replica value to scale up/down
● Which pods are scaled depends on RC selector
● Labels and selectors are used for grouping
● Can do quite complex things with RCs and labels

Example Replication Controller
apiVersion: v1
kind: ReplicationController
metadata:
name: nginx
spec:
replicas: 3
selector:
app: nginx
template:
metadata:
name: nginx
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx
ports:
- containerPort: 80

Replica Set
Replica Set is the next-generation Replication Controller. The only difference
between a Replica Set and a Replication Controller right now is the selector
support. Replica Set supports the new set-based selector which allow filtering
keys according to a set of values:
- In
- Notin
- exists (only the key identifier)
For example:
environment in (production, qa)
tier notin (frontend, backend)
partition
!partition

Deployments
A Deployment is responsible for creating
and updating instances of your
application
● Create a Deployment to bring up Pods and a
replica set.
● Check the status of a Deployment to see if it
succeeds or not.
● Later, update that Deployment to recreate the
Pods (for example, to use a new image).
● Rollback to an earlier Deployment revision if
the current Deployment isn’t stable.
● Pause and resume a Deployment.

Deployment example
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: nginx-deployment
spec:
replicas: 3
strategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 1
maxSurge: 1
minReadySeconds: 5
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:1.91
ports:
- containerPort: 80

Services
“defines a logical set of Pods and a
policy by which to access them”
● As Pods are ephemeral, we can't depend
on Pod IPs
● Services find pods that match certain
selection criteria
● Services can load balance between
multiple Pods
● Services can have a single IP that doesn’t
change

Services
A group of pods that act as one == Service
- group == selector
Defines access policy
- LoadBalanced, NodePort
Gets a stable virtual IP and Port
- Called the service portal
- Also a DNS name
- On prem additional loadbalancer is needed
VIP is captured by kube-proxy
- Watches the service consistency
- Updates when backend changes

Service example

Service example
apiVersion: v1
kind: Service
metadata:
name: railsapp
spec:
type: NodePort
selector:
app: railsapp
ports:
- name: http
nodePort: 36000
port: 80
protocol: TCP

Architecture
etcd (stores cluster state)
API Server
Scheduler
Controller manager
Kubelet (“node agent”)
Kube-proxy
Container Runtime
https://github.com/kubernetes/kubernetes/blob/release-1.3/docs/design/architecture.md

Architecture
Master Node (“Control Plane”)
Api server
- Point of interaction with the cluster
- Exposes an http endpoint
Controller Manager
- Responsible for most of the important stuff
- Interacts with the api server to retrieve cluster state
- Responsible for configuring networking
- Allocates node CIDRs
- Ensures correct number of pods are running
- Reacts to Nodes being added / deleted
- Manages Service Accounts and security tokens
Scheduler
- Schedules newly created pods to a Node

Architecture
Master Node (“Control Plane”)
Etcd
- Stores the state of the cluster
- Doesn’t necessarily have to be co-located with other components
- Must be backed up in a production scenario

kubelet
- Agent for running Pods
- Mounts volumes for Pods where required
- Reports the status of Pods back to rest of system
kube-proxy
- Enforces network rules on each Node (uses iptables)
- Responsible for forwarding packets to correct destination
Architecture
Worker Node

Master Node (api-server)
- Takes an argument for etcd servers
Master Node (controller-manager)
- Takes an argument for api server
- Creates/defines virtual networks for containers and services
- Takes an argument for cluster node CIDR
- Takes an argument for service CIDR
kubelet
- Configures the Docker bridge
- Takes an address for the cluster DNS
kube-proxy
- Takes an argument for the cluster node CIDR
Architecture
Networking

Architecture
Networking

AWS

Various service components:
- IaaS: EC2 / VPC
- PaaS: Elastic Beanstalk / ECS
- (No)SQL database services
- Data Storage / Warehousing / Processing
- Mobile Services
- Serverless Services
- CDN
AWS
Cloud Computing Platform

We will use CloudFormation to:
- Launch EC2 instances into an existing VPC
- Create a subnet for each kubernetes cluster
- Create a route table for each subnet
- Create Security Groups (firewall rules) for each cluster
- Create Autoscale Groups for Master and Worker nodes
AWS
Today: EC2, VPC and CloudFormation
Instance Configuration:
- Userdata: Instructions to be run by AWS cloud-init system after boot
- Chef: Userdata will instruct instances to bootstrap to Chef server
CloudFormation:
- Method of keeping Infrastructure as Code
- JSON based template that defines AWS Resources

AWS
Other ways to build
Getting Started guide: http://kubernetes.io/docs/getting-started-guides/aws/
- $ set=something ; wget something | bash
- Great for getting a cluster up and running quickly
- Inflexible for integration into existing VPCs
- Fussy if you put anything else in the VPC it creates
Kops: https://github.com/kubernetes/kops
- “kubectl for clusters”
- Will become the standard way to launch onto AWS
- Still in alpha
Run with your own: https://github.com/kelseyhightower/kubernetes-the-hard-way
- Takes some time
- Expect to reverse-engineer
- You will know exactly how the cluster is put together

Using the --cloud-provider=aws flag, the kubernetes components can be instructed
to leverage AWS IaaS features.
Master instances (running controller-manager) must have an appropriate IAM role
assigned.
Kubernetes can then
- Create and destroy Elastic Load Balancers (ELBs)
- Add and delete routes from cluster Route Table
- Add and delete firewall rules on cluster Security Group
AWS and Kubernetes
Kubernetes is able to configure AWS
Relevant resources must be appropriately tagged:
- Name: KubernetesCluster
- Value: ClusterId

AWS and Kubernetes
Our Workshop Architecture:
Network

AWS and Kubernetes
Our Workshop Architecture:
Servers

Hands-On

Build a cluster
● Choose yourself an ID for the cluster
$ git clone https://github.com/ContainerSolutions/kubernetes-aws-workshop.git
$ cd kubernetes-aws-workshop/
$ ./build [user-id]

Configure kubectl
$ eval `ssh-agent`
$ ssh-add /path/to/private.key
$ ./find-master [user-id]
x.x.x.x
$ ./set-cluster x.x.x.x
$ kubectl config view

Check the cluster status
$ kubectl cluster-info
$ kubectl get cs (componentstatus)
$ kubectl get nodes
$ kubectl get events
$ kubectl describe nodes

Deploy a container
$ kubectl create -f kube-files/nginx-pod.yml
$ kubectl get pods
$ kubectl describe pod nginx
# note the pod ip address

Create a service
$ kubectl create -f kube-files/nginx-service.yml
$ kubectl get svc
$ kubectl describe service nginx-service
# note the Endpoints
# note the IP
# note the NodePort

Investigate the service
$ kubectl describe service nginx-service
Name: nginx-service
Namespace: default
Labels: <none>
Selector: app=nginx
Type: NodePort
IP: 10.20.32.218
Port: http 80/TCP
NodePort: http 31975/TCP
Endpoints: 10.100.0.2:80
Session Affinity: None
$ ./run-nodes [user-id] curl -s [IP]
$ ./run-nodes [user-id] curl -s [Endpoints]
$ ./run-nodes [user-id] curl -s 127.0.0.1:[NodePort]

What’s happening?
$ ./find-nodes [cluster-id]
x.x.x.x
x.x.x.x
$ ssh ubuntu@x.x.x.x
$ ip route list
$ route -n
$ sudo iptables -L -t nat
# view route table in AWS, note that the pod CIDRs are routed directly to an EC2 NIC

Cluster Add-Ons
$ kubectl create -f kube-files/kubernetes-dashboard.yml
$ kubectl proxy
Starting to serve on 127.0.0.1:8001
# Go to 127.0.0.1:8001/ui

Cluster Add-Ons
$ kubectl create -f kube-files/kube-dns.yml
$ kubectl config use-context system
$ kubectl get pods
# Note the pods you’ve not seen yet. These are running cluster services
$ kubectl config use-context workshop

Observing DNS
$ kubectl create -f kube-files/busybox.yml
$ kubectl exec -ti busybox sh
# nslookup google.com
# nslookup nginx-service
# nslookup kubernetes-dashboard.kube-system
# cat /etc/resolv.conf
# exit

Deploying a service
$ kubectl delete pod nginx
$ kubectl delete svc nginx-service
$ kubectl create -f kube-files/nginx-deployment
$ kubectl get pods
$ kubectl get rs (replicaset)
$ kubectl delete pod [nginx-pod]
$ kubectl get pods

Deploying a service
$ kubectl expose deployment nginx --type=LoadBalancer
$ kubectl get svc -o wide
# ...wait

Deploying a microservice application
$ kubectl create -f kube-files/microservices-demo.yml
$ kubectl get svc -o wide
# ...wait

Tidy up...
$ kubectl delete service nginx
$ kubectl delete deployment nginx
$ kubectl delete -f kube-files/microservices-demo.yml
$ ./delete [user-id]
$ ssh-agent -k

Questions?
grant.ellis@container-solutions.com

Kubernetes on AWS

More Related Content

What's hot (20)

Similar to Kubernetes on AWS (20)

Recently uploaded (20)

Kubernetes on AWS