Continuous Integration Made Easy:
An Introduction with GitLab
Koundinya N V S S, Senior Data Scientist
Agenda
GitLab CI Basics
Overview of Continuous Integration (CI)
Building Jobs in GitLab CI
Creating and configuring GitLab CI jobs
Triggering builds and managing dependencies
GitLab CI Pipeline
Introduction to GitLab CI Pipeline
Creating and managing GitLab CI Pipeline jobs
Understanding DevOps
DevOps – Development + Operations
A set of practices and cultural values that ensures collaboration between development and operations teams.
Why?
To make software production and deployment faster in a repeatable and automated way in a cross-functional
environment
Source: https://www.simplilearn.com/tutorials/devops-tutorial/continuous-integration
Understanding DevOps – Continuous Methodologies
How?
Using Agile – an iterative approach
Continuous Methodologies
Understanding Continuous Methodologies
Continuous Integration (CI) – Automated testing and artifact creation
Continuous Delivery (CD) – Automated deployment to test and staging environments; Manual deployment to production
Continuous Deployment (CD) – Automated deployment to production
Continuous Monitoring – Automated monitoring and alert mechanisms
Continuous Integration
For every change pushed to the repository, the application is built, tested, and deployed for review
automatically and continuously.
Continuous Integration
Why?
Detect development problems soon
Reduce risks of cost, schedule, and budget
Find and remove bugs earlier
Deliver new features and get user feedback more rapidly
How?
Maintain a single source repository
Automate the build
Make your build self-testing
Keep the build on the CI machine
Pace up the build with periodic commits
Test in a clone of the production (staging) environment
Make it easy for everyone to get the latest executable
Make the process transparent for everyone
Continuous Delivery/Deployment
CI/CD: For every change pushed to the repository, the application is built, tested, and deployed for
production automatically and continuously.
Continuous Delivery/Deployment
Why?
Reduce deployment risks
Change the version in production more rapidly
Get the feedback earlier
How?
Continuously integrating the software done by the development team
Run automated tests
Push build to the production-like environment
Can release one version at the push of the button
Understanding DevOps – Summary of Continuous Methodologies
Source: https://biplus.com.vn/ci-cd-devops/
Software/Packages for Continuous Integration
GitLab GitLab Jenkins
Jenkins Pros Pros
Circle CI Parallel builds Highly customizable
Travis CI Native Docker support Community. A lot of resources and tutorials
Fully Integrated in Gitlab Supports multiple version control systems
Bamboo
Configurability per branch already on jobs-level Cross-platform
Spinnaker
Permission inheritance (Repository Manager) Build-in time based execution
AWS Code Pipeline
Multi-language UI and Dashboard
Azure DevOps
One application Report integration
Cons Cons
Integrated only in Gitlab Poor quality of some plug-ins
High overhead (setup and host)
Why GitLab? GitLab Features
Source: https://about.gitlab.com/blog/2023/07/27/gitlab-flow-duo/
CI/CD Life Cycle
Source: https://www.inviggo.com/blog/an-insiders-guide-to-ci-cd-for-mobile-app-development-tools-expert-tips
GitLab CI/CD Common Terms
The `.gitlab-ci.yml` file
Jobs and Stages
Runners
Pipelines
CI/CD variables
CI/CD components
The `.gitlab-ci.yml` File
contains the configuration for your CI/CD pipeline
Example
usually present at the root of your project
can name this file anything you want, but `.gitlab-ci.yml` is the most common name
In this file, you can define:
The tasks you want to complete, for example, test and deploy your application
Other configuration files and templates you want to include
Dependencies and caches
The commands you want to run in sequence and those you want to run in parallel
The location to deploy your application to
Whether you want to run the scripts automatically or trigger any of them manually
Jobs and Stages
Jobs Example
Jobs define “what to do”
Building blocks of GitLab CI Pipelines
Executed by Runners
Can be configured with scripts, commands, and artifacts
must contain at least the ‘script’ clause
Stages define “when to run”
Help organize jobs sequentially to create GitLab CI Pipelines
Stages Example
represent different phases of the software delivery process
Multiple jobs in the same stage are executed in parallel
If all jobs in a stage succeed, the pipeline moves on to the next stage
If any job in a stage fails, the next stage is not (usually) executed and the pipeline ends early
Why multiple jobs?
The smaller a job the easier parallelization across multiple runners is
Smaller jobs are easier to diagnose when they break
Pipelines
Collections of jobs and stages that define the software delivery How to trigger pipelines?
process
Manually: Trigger on demand for specific
Each pipeline represents a complete build, test, and deployment builds or deployments.
cycle
Automatically: Trigger on push/merge
Executed automatically when triggered according to defined events, schedule, or API calls.
conditions
Conditional triggering: Use rules to
specify when to trigger pipelines based on
specific conditions.
Types of pipelines
Basic Merged results
Directed Acyclic Merge trains
Graph (DAG)
Parent-child
Merge request
Multi-project
CI/CD Variables
Control the behavior of jobs and pipelines Example
Store values you want to re-use
Avoid hard-coding values (such as credentials, API keys, or other
secrets) in your `.gitlab-ci.yml` file
Why CI/CD variables?
Easy configuration changes without modifying pipeline code
Enhance security and maintainability by centralizing access control
Dynamic configuration based on environment or user needs
Simplified collaboration and sharing between team members
To make variables more secure, consider
using external secrets and file type variables
Image Source: https://about.gitlab.com/blog/2021/04/09/demystifying-ci-cd-variables/
CI/CD Components
Reusable units of pipeline configuration that encapsulate specific functionalities
Promote modularity, reusability, and maintainability in complex pipelines
Can be shared across projects and repositories within the GitLab instance
Can take input parameters for dynamic behavior and customization
Why CI/CD Components?
Reduce code duplication and boilerplate code in pipelines
Encourage collaboration and sharing of reusable components within teams
Enable centralized management and updates for commonly used functionalities
Simplify complex pipelines by breaking them down into smaller, manageable units
Runners
Software agents that execute CI/CD jobs on the infrastructure
Runners build, test, and deploy jobs as instructed by GitLab CI
Can be self-hosted on your servers, cloud VMs, or containers
An executor is a tool for the runner that determines the environment each job runs in
Runner Installations Supported Executors
• SSH • Docker
• Shell • Docker
• Parallels Autoscaler (Beta)
• VirtualBox • Docker Machine
• Kubernetes (auto-scaling)
• Instance (Beta) • Custom
CI/CD YAML Syntax Reference
GitLab CI/CD pipeline configuration includes
Global Keywords – configures pipeline behaviour
Header Keywords – define specifications for
external configuration files
Job Keywords – define the behaviour of individual
jobs within a pipeline
How to set up a GitLab CI/CD Pipeline?
Step 1: Define what to run
Step 2: Define where to run
Step 3: Give it a go!
CI/CD Basic Pipeline Example
Source: https://docs.gitlab.com/ee/ci/pipelines/pipeline_architectures.html#basic-pipelines
