Intro

12 October 2022 18:13

A Google Cloud project is an organizing entity for your Google Cloud resources.

Virtualization and cloud computing waves:

---------- Colocation (physical servers)--------> virtualized data centre (VMs) ----------> container based architecture

IaaS raw compute, storage, networking

Google's resource manager tool:
PaaS libraries and runtimes that run on IaaS

Pricing and billing:

Budget, alerts, reports, quotas(protects over-consumption of resources)

Quotas:
Rate quota --> resets after a specific time
Allocation quota --> governs number of resources

Project:
Project ID, Project number- assigned by google, globally unique, immutable
Project name - chosen by customer, mutable

GCP-Cloud Engineer Page 1

Project:
Project ID, Project number- assigned by google, globally unique, immutable
Project name - chosen by customer, mutable

Folder:
Folder can contain other folders, projects

Organization Node: Cloud SDK : all below tools are under bin directory
Org. policy administrator, Project Creator gsutil --> command line tool for cloud storage
gcloud --> command line tool for google cloud products/services
bq --> command line tool for BigQuery

Cloud Shell :
Command line access from browser
Debian based VM with 5 GB persistent home dir

GCP-Cloud Engineer Page 2

Links to resources
13 October 2022 14:02

[Link] -gcp-a-decision-tree

[Link]

[Link]

[Link]

GCP-Cloud Engineer Page 3

Compute
13 October 2022 17:38

Compute Engine: Shielded VMs are hardened virtual machines that use Secure Boot, virtual trusted
Billed per second, 1 minute minimum platform module enabled Measured Boot, and integrity monitoring.
Sustained use discounts / committed use discounts

Sole tenant nodes:

If you need GPU support for a compute-heavy workload, you can choose to attach On a sole tenant node, only VMs from the same project will run on that node.
GPUs to certain machine types. You can only use GPUs with general-purpose N1 They do not need to use the same operating system.
VMs or accelerator-optimized A2 VMs. Availability of these machine types varies Sole tenant nodes are not restricted to a single VM.
by zone, so make sure you pick a zone that has GPU capability.

Normally the boot disk defaults to being deleted automatically when the instance is
deleted. But sometimes you will want to override this behaviour. This feature is very
important because you cannot create an image from a boot disk when it is attached to a
running instance.

So you would need to disable Delete boot disk when instance is deleted to enable
creating a system image from the boot disk.

Machine family > Machine Series > Machine Type

GCP-Cloud Engineer Page 4

GCP-Cloud Engineer Page 5
How to choose Compute option ?

create VM (enable keep the disk option) --> install Apache web server and configure it to boot after VM start/restart --> reset (stop and reboot) the VM and verify the
web server is up --> then delete the VM --> verify disk is still available -- > create image from the disk --> delete the disk

An instance template is an API resource that you can use to create VM instances and managed instance groups. Instance templates define the machine type, boot disk
image, subnet, labels, and other instance properties.

Managed instance group health checks proactively signal to delete and recreate instances that become unhealthy.

Different autoscaling metric options:

CPU utilization, HTTP load balancing utilization, Cloud pub/sub queue, Cloud monitoring metric

GCP-Cloud Engineer Page 6

Data Storage
13 October 2022 17:48

BigQuery data transfer --> To connect your web host (VM instance hosting your web app) to cloud SQL, add
Cloud Storage to BigQuery data transfer, free service, done using a single command a network (IP or set of IP of your host), in the Connections tab.

Try to put web app host and SQL instance in the same region and zone to achieve
Cloud Spanner: best performance
SQL relational DBMS with joins and secondary indices, HA, strong global consistency, DB size > 2 TB, high
number of IOPS
Bigtable is based on HBASE API. Check out !
Firestore:
Automatic multi-region data replication, strong consistency guarantee, atomic batch operations, real
transaction support

Bigtable:
Powers google maps, gmail, google search, google analytics

GCP-Cloud Engineer Page 7

GCP-Cloud Engineer Page 8
Networking
13 October 2022 18:15

Proxy and pass-through load balancers

In Google Cloud, load balancers can be proxied or pass-through. Proxied load balancers terminate
connections and proxy them to new connections internally. Pass-through load balancers pass the
connections directly to the backends.

GCP-Cloud Engineer Page 9

IAM
14 October 2022 10:30

3 types of Role:

Basic -- owner / editor / viewer / billing admin

Predefined
Custom -- you need to manage permissions for roles that you have defined, can be applied
only at project OR organizational level

Cloud Identity:
Organizations can define policies and manage their users and groups using Google Cloud Console

• Folders are used to group resources that share common IAM policies.
• Service accounts are specific to a set of operating requirements within a project.
• Permissions are associated with roles but not directly with folders.
• IAM roles are granted to identities, not folders.

Use cloud identity/workspace to create and managers users/groups.

Use Cloud Identity to configure SAML SSO,

If SAML2 isn't supported, use 3rd party solutions like Okta, Ping etc.

GCP-Cloud Engineer Page 10

GCP-Cloud Engineer Page 11
 VPC
28 October 2022 21:26

VPC - create a private environment within a public cloud infrastructure

VPC combines scalability of public cloud with privacy of private cloud (private data centre)

Google VPC networks are global and can have subnets in any google cloud region worldwide

A shared VPC allows projects to share a common VPC network.

Shared VPC can happen only within a single organization.
If you want to connect VPC networks across organizations, implement VPC
network peering.

Cloud Router: (uses BGP protocol)

Cloud Router automatically learns new subnet IP address ranges in your VPC
network and can announce them to your peer network (on-prem or some
other network)

The default network has a subnet in each Google Cloud region

Networks are global, and subnets are regional.

VPC creation in auto mode -- subnets are created in each region automatically
Deny-all-ingress and allow-all-egress have lower priority (higher integers)

Different ways of connecting your network to VPC:

1. IPSec VPN protocol (VPN)
2. Direct peering
3. Carrier peering
4. Dedicated Interconnect (SLA available)
5. Partner Interconnect

A project contains upto 5 networks that can be

shared/peered.

Auto mode networks can be converted to custom mode

but not vice-versa.

First IP is reserved for gateway,

e.g. in [Link]/20 block, [Link] is reserved.

• Every subnet has 4 reserved IP addresses (first 2 and last 2)

• Auto mode can be expanded from /20 to /16 but not shrinked
• External IP addresses are ephemeral by default and optional

Routes -
map an IP range to a destination
are created when a subnet is created
have destination mentioned in CIDR notation
must match with firewall rules to deliver the traffic

Cloud Firewall Rules

Every VPC network has two implied firewall rules that block all
incoming connections and allow all outgoing connections.

you can ping VM's internal IP because of allow-custom rule and

you can ping VM's external IP because of allow-icmp rule

Recommended: use custom mode networks in prod

VPC networks are by default isolated private networking domains.

Therefore, no internal IP address communication is allowed between
networks, unless you set up mechanisms such as VPC peering or VPN.

GCP-Cloud Engineer Page 12

Cloud NAT:
Google's managed NAT service
Best practice:
assign only internal IP addresses to VM's and use NAT in front
of them

private google access allows VM's with only internal IP addresses to

communicate with google APIs and services (with public IP addresses)
e.g. in adjoining example, A1 does not have public IP but still can access
google APIs and services (since private google access is on for subnet-
a). B1 has neither public IP nor private google access enabled, hence
cannot access either internet or google services and APIs.

You can SSH into a VM instance that does not have external IP address using IAP
(identity aware proxy) tunnel.

When instances do not have external IP addresses, they can only be reached by
other instances on the network via a managed VPN gateway or via a Cloud IAP
tunnel. Cloud IAP enables context-aware access to VMs via SSH and RDP without
bastion hosts. To learn more about this, see the blog post Cloud IAP enables context-
aware access to VMs via SSH and RDP without bastion hosts .

IAP uses your existing project roles and permissions when you connect to VM
instances. By default, instance owners are the only users that have the IAP Secured
Tunnel User role.

Private Google Access is enabled at the subnet level. When it is enabled,

instances in the subnet that only have private IP addresses can send traffic to
Google APIs and services through the default route ([Link]/0) with a next hop
to the default internet gateway. But this still does not enable the internal VM
to access internet. We need NAT for it.

The Cloud NAT gateway implements outbound NAT, but not inbound
NAT. In other words, hosts outside of your VPC network can only
respond to connections initiated by your instances; they cannot initiate
their own, new connections to your instances via NAT.

Cloud NAT lets your VM instances and container pods communicate

with the internet using a shared, public IP address.
Cloud NAT uses Cloud NAT gateway to manage those connections.
Cloud NAT gateway is regional and VPC network specific. If you have
VM instances in multiple regions, you’ll need to create a Cloud NAT
gateway for each region.

GCP-Cloud Engineer Page 13

GCP-Cloud Engineer Page 14
Logging/Monitoring
30 October 2022 13:39

Constraints are the standard way to restrict where resources can be created and applying policies with constraints
Cloud Audit Logs maintain three audit logs: will enforce those constraints for all resources in the organization. If the policy were applied at the folder level, it
Admin Activity logs would have to be applied for all folders and that is not as efficient as applying at the organization level.
Data Access logs
System Event logs. Four golden signals: Latency, Traffic, Saturation, Errors

Latency indicators: Traffic indicators: Saturation: Errors:

Page load latency, # HTTP requests per second % memory utilization Wrong answers/incorrect content
Query duration, # requests for static vs. dynamic content % thread pool utilization # 400/500 HTTP codes
Service response time, Network I/O % cache utilization # failed requests
Transaction duration, # concurrent sessions % cpu utilization # exceptions
Time to first response, # transactions per second % disk utilization # stack traces
Time to complete data return # retrievals per second Disk quota Servers that fail liveness checks
# active requests Memory quota # dropped connections
# write ops, # read ops # available connections
# active connections # users on the system

Types of logs

GCP-Cloud Engineer Page 15

 Export logs from cloud logging to BigQuery to analyze and then
visualize in Data Studio
Good practice to install logging and monitoring agents on all VMs

You want to be notified if your application is down -- uptime check

You’re setting up a load balancer for an application you deployed to Compute Engine. To ensure the load balancer only sends requests to machines that are working;
you would use what GCP tool. ---- health check

Installing Monitoring and Logging agents in VMs is recommended

Google's best practices for image creation:

Can create custom metrics (in addition to what GCP already provides):
Either use cloud monitoring API
OR
Use OpenCensus

There are few resources that generate a lot of logs, e.g. logging agent on VM instance, cloud load balancer. We can exclude those from our logs view.
Common logs that we can exclude are:
Load balancer (90%)
VPC flow logs (percentages, CIDR)
HTTP 200 OK messages

GCP-Cloud Engineer Page 16

GCP-Cloud Engineer Page 17
Cloud DNS
30 October 2022 13:43

Google's free DNS service - [Link] Cloud Identity provides domain verification records, which are added to DNS settings for the
domain.
Cloud DNS - 100% uptime SLA

Google Cloud alias IP ranges let you assign ranges of internal IP addresses as aliases to a
virtual machine's (VM) network interfaces. This is useful if you have multiple services
running on a VM and you want to assign each service a different IP address. Alias IP
ranges also work with GKE Pods.

If you have only one service running on a VM, you can reference it by using the
interface's primary IP address. If you have multiple services running on a VM, you might
want to assign each one a different internal IP address.

GCP-Cloud Engineer Page 18

Persistent disks
30 October 2022 13:45

Clone a persistent disk:

The source and cloned disk must be in the same zone and region and must be of the same type.
The size of the clone must be at least the size of the source disk.

GCP-Cloud Engineer Page 19

General
30 October 2022 13:49

URL maps specify direct requests to particular services.

Routes are used to specify paths to destination IP addresses outside a subnet.
Firewall rules control the flow of traffic on a network.
Traces are used to understand performance characteristics of services in a distributed system.

Cloud Source Repositories:

Google hosted GIT repos, integrated with app engine/compute engine/debugger/error reporting

Rate quotas: resets at regular intervals, e.g. 1000 API calls per 100 seconds
Allocation quotas: e.g. 5 VPC networks per project

Google Recommended practice: labels are attached to resources and

these labels are then propagated into billing items.

GCP-Cloud Engineer Page 20

Cloud load balancing
30 October 2022 15:07

Google Cloud HTTP(S) load balancing is implemented at the edge of Google's network in Google's
points of presence (POP) around the world. User traffic directed to an HTTP(S) load balancer enters
the POP closest to the user and is then load-balanced over Google's global network to the closest
backend that has sufficient available capacity.

GCP-Cloud Engineer Page 21

CDN
30 October 2022 15:11
Cloud CDN cache modes:
Cache mode control the factors that determine whether or not
If you are using CDN services of 3rd party, chances are they ae already part of google CDN Cloud CDN caches your content
interconnect program. So, you can continue using the same 3rd party service.
USE_ORIGIN_HEADERS
CACHE_ALL_STATIC
FORCE_CACHE_ALL

GCP-Cloud Engineer Page 22

Cloud Storage
30 October 2022 16:16

Security:

Scope = who, permission = what

bucket level permissions - IAM

object level access control - ACLs

GCP-Cloud Engineer Page 23

Cloud RUN
31 October 2022 10:27
Container registry is now called artifact registry

GCP-Cloud Engineer Page 24

Anthos
31 October 2022 10:43

Hybrid and multi-cloud solution

Framework rests on Kubernetes and GKE on-prem
Provides a rich set of tools for monitoring and maintenance

Migrate For Anthos: below architecture depicts how migrate for Anthos works step by step

Migrate for compute engine: bring your application in on-prem/non-gcp cloud into VMs in google cloud

GCP-Cloud Engineer Page 25

App Engine
31 October 2022 11:25

Standard:
Persistent storage with queries, sorting and transactions
Auto scaling and load balancing
Async task queues for performing work outside the scope of a request
Scheduled tasks for triggering events at specified times or regular intervals
Integration with other GCP services and API

GCP-Cloud Engineer Page 26

API management tools
31 October 2022 11:58

APIgee Edge
Cloud endpoints

GCP-Cloud Engineer Page 27

Cloud Functions
31 October 2022 13:25

GCP-Cloud Engineer Page 28

Cloud shell
31 October 2022 17:00

The gcloud command often requires you to specify values such as a Region, Zone, or Project ID. Entering them
repeatedly increases the chance of making typing errors. If you use Cloud Shell frequently, you may want to set common
values in environment variables and use them instead of typing the actual values.
You can use environment variables like this in gcloud commands to reduce the opportunities for typos and so that you
won't have to remember a lot of detailed information

INFRACLASS_REGION= asia-east1

Add env variables (project id, name, region, zone etc.) in a config file
and add below command to .profile (bash profile) so these variables are
always loaded when you open up cloud shell:

source some_dir_name/config

GCP-Cloud Engineer Page 29

gcloud commands
01 November 2022 11:05

create a VPC network

gcloud compute networks create managementnet --project=qwiklabs-gcp-03-9f64a1c6b0f0 --subnet-mode=custom --mtu=1460 --bgp-routing-mode=regional

create a subnet
gcloud compute networks subnets create managementsubnet-us --project=qwiklabs-gcp-03-9f64a1c6b0f0 --range=[Link]/20 --stack-type=IPV4_ONLY --
network=managementnet --region=us-central1

create a firewall rule

gcloud compute --project=qwiklabs-gcp-03-9f64a1c6b0f0 firewall-rules create managementnet-allow-icmp-ssh-rdp --direction=INGRESS --priority=1000 --network=managementnet --
action=ALLOW --rules=tcp:22,tcp:3389,icmp --source-ranges=[Link]/0

create a VM
gcloud compute instances create privatenet-us-vm --zone=us-central1-c --machine-type=f1-micro --subnet=privatesubnet-us --image-family=debian-10 --image-project=debian-cloud --
boot-disk-size=10GB --boot-disk-type=pd-standard --boot-disk-device-name=privatenet-us-vm

Move a VM from one zone to another

gcloud compute instances move

GCP-Cloud Engineer Page 30

IAM best practises
02 November 2022 13:03

GCP-Cloud Engineer Page 31

Cloud SQL
02 November 2022 15:56

GCP-Cloud Engineer Page 32

Resource Manager
02 November 2022 19:12

Child policies cannot restrict access granted at the parent level

While querying the bigquery, you need to specify DATASET_ID and

TABLE_ID. PROJECT_ID is optional.

GCP-Cloud Engineer Page 33

GCP-Cloud Engineer Page 34
Hybrid Connectivity and shared VPC
09 November 2022 21:35

Connect your on-prem network to your GCP VPC network over public internet using IPSec tunnel
Supports both static and dynamic routes (need to configure cloud router which uses BGP)

Features:
Regional resource
Good for low volume data connections
Traffic encrypted by one VPN gateway and decrypted by the other VPN gateway
99.9% SLA
Supports: site-to-site VPN, static routes, dynamic routes (cloud router), IKEv1/IKEv2 ciphers

[Link]

Supports site-2-site VPN connection for below different scenarios:

GCP-Cloud Engineer Page 35

Problem: how to handle growing network (newly added subnets in the VPC or on-prem) ?
Answer: use Cloud Router which uses BGP session

GCP-Cloud Engineer Page 36

HA VPN is a regional resource and cloud router that by default only sees the routes in the region
in which it is deployed. To reach instances in a different region than the cloud router, you need to
enable global routing mode for the VPC. This allows the cloud router to see and advertise routes
from other regions.

Layer 2 uses VLAN technology

GCP-Cloud Engineer Page 37

Direct Interconnect has SLA but Direct peering does not have.

GCP-Cloud Engineer Page 38

 Web app server and R, P, A services are part of shared
VPC. Web app server can communicate with R, P, A services
Using internal IP addresses. Web app server connects with
Clients/on-prem using external IP addresses.

Producer network needs to peer with Consumer

And vice versa.

GCP-Cloud Engineer Page 39

Managed Data Analytics Services
09 November 2022 22:42

Checkout Trifacta

GCP-Cloud Engineer Page 40

Dataproc : create hadoop, spark clusters with 90 seconds or less on average, whereas normally it can take about 5-30 minutes

You can see VMs that are created (master and workers) in the cluster, SSH into master

GCP-Cloud Engineer Page 41

Load balancing and autoscaling
10 November 2022 14:21

Global load balancers:

HTTP(S), SSL proxy, TCP proxy
Regional load balancers:
Internal TCP/UDP
Network TCP/UDP
Internal HTTP(S)

Regional internal load balancers use Andromeda (Google's SDN network virtualization stack)
Regional network load balancers use Maglev (Google's NLB, large distributed software that
runs on commodity hardware)

Regional managed instance groups are preferred over zonal managed instance groups
as your instance groups are not restricted to a zone OR you do not need to manage
multiple zonal instance groups.

Stateful and stateless MIGs

Load balancers:
URL maps -- some URLs are mapped to a set of instances and some others to another set of
instances

Architecture of HTTP(S) load balancer:

Global forwarding rule forwards the incoming requests to an HTTP

proxy which checks the requests in URL map to decide which backend
service should handle the requests.

GCP-Cloud Engineer Page 42

Content based load balancing -- e.g. /audio routed to audio backend Only HTTP(S), SSL proxy, TCP proxy supports IPV6 traffic.
service, /video routed to video backend and so on

Network endpoint group:

GCP-Cloud Engineer Page 43

 Normally, there are 2 connections. Client instance to LB and
LB instantiates a new connection to backend instance.
But GCP uses Andromeda to directly transmit traffic from
client to backend instance.

GCP-Cloud Engineer Page 44

GCP-Cloud Engineer Page 45
IaC
11 November 2022 17:14

Terraform:
Infrastructure automation tool
Repeatable deployment process
Focus on the application
Parallel deployment
Template-driven

Hashicorp Configuration Language (HCL)

Cloud Foundation Toolkit

create multiple VMs with count meta-argument Declarative vs Imperative approach to infrastructure:
Imperative: Give me 5 servers (may lead to repeated creation of 5 servers)
Declarative: I should have 5 servers (always compares desired vs current state)

Dependency graph
Implicit Vs Explicit dependency

Depends on meta argument

GCP-Cloud Engineer Page 46

output values are like return values in programming languages

GCP-Cloud Engineer Page 47

Containers and K8S
12 November 2022 09:21

Application + Dependencies = Image

Containers use a varied set of technologies:

Processes - To give a container its own virtual memory address
space
Linux namespaces - To control an application's ability to see parts
of the directory tree and IP addresses
Cgroups - To control an application's maximum consumption of
CPU time and memory
union file systems - To efficiently encapsulate applications and
their dependencies into a set of clean, minimal layers

K8S:
K8S 'watch loop' to bring the system to its desired state and maintain it there

Kubernetes objects: persistent entities representing the state of the cluster

Object spec: desired state described by us
Object status: current state described by Kubernetes

If a pod contains more than 1 container, they are tightly coupled and share networking and storage space within the pod.
Each pod has a unique IP assigned.

etcd: cluster's datastore

Kubelet: a k8s agent on each node. Kube-api server connects with kubelet for launching a pod, for example.
Kube-proxy: maintains network connectivity amongst pods in a cluster.

Pods do not auto-heal. So, better to user controller objects (Deployment, StatefulSet, DaemonSet, Job)
e.g. you want 3 NGINX servers up and running all the time, instead of creating 3 pods, create a Deployment which
creates a ReplicaSet object which manages the desired state of 3 running NGINX pods.

You will work with Deployment objects directly much more often than ReplicaSet objects. But it's still helpful to
know about ReplicaSets, so that you can better understand how Deployments work. For example, one capability of
a Deployment is to allow a rolling upgrade of the Pods it manages. To perform the upgrade, the Deployment object
will create a second ReplicaSet object, and then increase the number of (upgraded) Pods in the second ReplicaSet
while it decreases the number in the first ReplicaSet.

How to manage underlying hardware allocation amongst the K8S objects ?

answer: abstract underlying physical hardware into logical units called namespaces

Apply namespace at the command line level

Deployment: ensures that a defined set of pods is running at any given time

GCP-Cloud Engineer Page 48

How to manage underlying hardware allocation amongst the K8S objects ?
answer: abstract underlying physical hardware into logical units called namespaces

Apply namespace at the command line level

Deployment: ensures that a defined set of pods is running at any given time

Controller (e.g. Deployment Controller) is a k8s loop process that makes sure the observed
state of the cluster matches the desired state of the cluster (by creating/deploying necessary
pods, for example.)

K8S Deployment has 3 different lifecycle states:

Progressing state - the task is being performed, e.g. creating/updating a new ReplicaSet
Complete state - e.g. all ReplicaSets are updated to latest versions and old ReplicaSets are removed
Failed State - e.g. creation of a new ReplicaSet could not be completed

GCP-Cloud Engineer Page 49

Session affinity - let client's first request to determine Volumes are attached to pods.
which pod will be used for subsequent connections Ephemeral and persistent volumes

apiVersion: v1 Whenever a pod is added to a node, emptyDir is created. It is stored in a local volume of a
kind: Service node. So, even if a container crashes, the emptyDir is safe. But if a pod is removed, data
metadata: stored in emptyDir is gone.
name: nginx
spec:
type: LoadBalancer
sessionAffinity: ClientIP
selector:
app: nginx
ports:
- protocol: TCP
port: 60000
targetPort: 80

A pod has an IP address and all containers within it share it. E.g. a legacy app is running in a container (port 8000) and us es an nginx reverse
proxy in another container in the same pod. nginx forwards inbound requests to [Link]:8000

GCP-Cloud Engineer Page 50

What the hell is PersistentVolumes and PersistentVolumeClaims ?????

PersistentVolumes are storage that is available to a Kubernetes cluster. PersistentVolumeClaims enable Pods to access PersistentVolumes.
Without PersistentVolumeClaims Pods are mostly ephemeral, so you should use PersistentVolumeClaims for any data that you expect to
survive Pod scaling, updating, or migrating.

GCP-Cloud Engineer Page 51

GKE
11 November 2022 17:26

GKE clusters are not exposed to internet. But they can be connected to authorized networks through external IP address OR other GCP services (e.g.
logging, monitoring)

GKE control plane : not exposed to gcp customers

GKE nodes: VM instances GCP users can manage

GCP-Cloud Engineer Page 52

Cloud Build
11 November 2022 22:31

Command to build docker container image from Dockerfile:

gcloud builds submit --tag [Link]/${GOOGLE_CLOUD_PROJECT}/quickstart-image .

Using custom build configuration in Cloud Build

The true power of custom build configuration files is their ability to perform other actions, in parallel or in sequence, in addition to
simply building containers: running tests on your newly built containers, pushing them to various destinations, and even deploying
them to Kubernetes Engine.

GCP-Cloud Engineer Page 53

Migrate for Anthos
12 November 2022 12:39

GCP-Cloud Engineer Page 54

GCP-Cloud Engineer Page 55
GCP-Cloud Engineer Page 56
GCP-Cloud Engineer Page 57
SRE
16 November 2022 16:37

Common business metrics: ROI, earnings before interest and tax(EBIT), employee turnover, customer churn
Common software metrics: Pageviews, User registration, Click-throughs, Checkouts

Metrics must be SMART(Specific, Measurable, Achievable, Relevant, Time-bound)

SLI -- a quantifiable measure of service reliability

SLO -- a reliability target for SLI
Reliability -- a measure to indicate whether an application performs its intended function for a given amount of time
SLO targets should be substantially higher than the reliability SLAs promise

Error budget -- SLOs imply a certain acceptable level of unreliability. This is a budget that can be allocated.
an error budget is the amount of error that your service can accumulate over a certain period of time before your users start being unhappy. You can think of it as the pain
tolerance for your users, but applied to a certain dimension of your service: availability, latency, and so forth.

SLO is supposed to be a line below which customers become unhappy.

example:
Choose SLI specification from the above menu, for example.
Availability -- the profile page should load successfully.
Latency -- the profile page should load quickly.

4 steps of SLO creation:

1. Choose an SLI specification
2. Substitute definitions to create a detailed SLI implementation
3. walkthrough user journey and look for coverage gaps
4. set aspirational SLOs based on business needs

SLI and error budget example:

Here’s an example. Imagine that you’re measuring the availability of your home page. The availability is measured by the amount of requests responded with an error,
divided by all the valid requests the home page receives, expressed as a percentage. If you decide that the objective of that availability is 99.9%, the error budget is 0.1%.
You can serve up to 0.1% of errors (preferably a bit less than 0.1%), and users will happily continue using the service.

GCP-Cloud Engineer Page 58

GCP-Cloud Engineer Page 59
GCP-Cloud Engineer Page 60
Advanced logging/monitoring
20 November 2022 09:34
Key access control roles for log-based metrics:
Logging/logs configuration writer:
list, create, get, update, delete log-based metrics
Logs viewer:
view existing metrics
Monitoring viewer:
read the timeseries in logs-based metrics
Logging Admin, Editor, Owner:
broad level roles that can create logs-based metrics

Log based metrics type:

Counter Vs Distribution

Dataflow for real time logs processing; if real-time capability not

required, we can directly export logs to bigquery from logging.

Also, archive logs for long term storage in Cloud Storage.

If you want logs to be exported to on-prem Splunk instance, use
pub/sub.

CLOUD LOGGING ARCHITECTURE:

GCP-Cloud Engineer Page 61

Incident Response:
Incident management roles:
Incident Commander (IC)
Communications lead (CL)
Users and stakeholders
Operations lead
Primary and secondary responders

Firewall rules logging enable (disable):

gcloud compute firewall-rules update <<name of firewall rule>> --enable-logging (OR --no-enable-logging)

GCP-Cloud Engineer Page 62

GCP-Cloud Engineer Page 63
 Data access logs can be set at ORG, FOLDER or PROJECT level.
Some are free tier, but some are chargeable.

GCP-Cloud Engineer Page 64

GCP-Cloud Engineer Page 65
Network Monitoring
20 November 2022 11:29

GCP-Cloud Engineer Page 66

Application Performance
20 November 2022 13:37

GCE, GKE, EXT systems need CLOUD DEBUGGER AGENT role in order to use debugger
Also, need access to source code location

Cloud Trace:
Each trace is a collection of Spans
A span wraps metrics about an application unit of work
a context, timing, and other metrics

GCP-Cloud Engineer Page 67

GCP-Cloud Engineer Page 68