More Related Content
Similar to Using Docker in CI process (20)
Using Docker in CI process
- 1. Introduction The pipeline The environment The images The problems Finish Using Docker in CI process Applied Docker G. Godlewski March 16, 2016
- 2. Introduction The pipeline The environment The images The problems Finish Author About me Programming, karate, music and cheese making. Currently working for SMT Software Services
- 3. Introduction The pipeline The environment The images The problems Finish The background Long time ago, in a galaxy far, far away...
- 4. Introduction The pipeline The environment The images The problems Finish The background
- 5. Introduction The pipeline The environment The images The problems Finish The background Scope of automation Unit tests Functional tests Integration tests Quality checks (nightly build, code coverage, phpmd, phpcs, jshint, jslint, ...) Deployments One should constantly search for spots where time could be saved
- 6. Introduction The pipeline The environment The images The problems Finish Definitions Continuous Integration Martin Fowler: Continuous Integration is a software development practice where members of a team integrate their work frequently, usually each person integrates at least daily - leading to multiple integrations per day. Each integration is verified by an automated build (including test) to detect integration errors as quickly as possible.
- 7. Introduction The pipeline The environment The images The problems Finish Definitions Continuous Delivery Martin Fowler: A common question we hear is “what is the difference between Continuous Delivery and Continuous Deployment?” Both terms are similar and were coined around the same time. I see the difference as a business decision about frequency of deployment into production. Continuous Delivery is about keeping your application in a state where it is always able to deploy into production. Continuous Deployment is actually deploying every change into production, every day or more frequently.
- 8. Introduction The pipeline The environment The images The problems Finish Product The point of view
- 9. Introduction The pipeline The environment The images The problems Finish Process The process Building - parts put together, diagnostics and verification, removal of unnecessary parts Packaging - a ready product is being put into a package (container) Delivery - the whole package is shipped to the client in a clearly defined manner Each of the stages has a clear I/O definition
- 10. Introduction The pipeline The environment The images The problems Finish Process Building up a complete solution In case of a product composed from several components (eg. complex system) we could use the analogy of buying kitchen furniture. All packages contain components that fulfill a defined set of criteria We know how the components should be connected We know how the components should be placed within client’s infrastructure (kitchen) Don’t dismiss the power of analogy!
- 11. Introduction The pipeline The environment The images The problems Finish Warming up! What will we use Docker for Performing CI builds within isolated containers (docker in docker, sic!) Providing the package for the product in which will it be further distributed Running our applications
- 12. Introduction The pipeline The environment The images The problems Finish Warming up!
- 13. Introduction The pipeline The environment The images The problems Finish Warming up! Getting ready 1 Ensure the right kernel version (starting from 3.10!) 2 Ensure you have all required kernel modules (aufs, devicemapper etc - depends on the distribution) 3 Latest docker-engine installed 4 Latest docker-compose installed
- 14. Introduction The pipeline The environment The images The problems Finish Additional tools Ansible What for? Describe how the application should be delivered (deployment to server) Configuration management Why? Great documentation Huge amount of modules - I didn’t have to build my own yet!
- 15. Introduction The pipeline The environment The images The problems Finish Own Docker Hub Docker Hub hub: restart: always image: registry:2 ports: - "5000:5000" environment: TERM: linux REGISTRY_HTTP_TLS_CERTIFICATE: :) REGISTRY_HTTP_TLS_KEY: :) REGISTRY_AUTH: htpasswd REGISTRY_AUTH_HTPASSWD_PATH: /auth/htpasswd REGISTRY_AUTH_HTPASSWD_REALM: Registry Realm volumes: - "/home/docker-distro/registry/data:/var/lib/registry" - "/etc/ssl/certs:/certs" - "/etc/ssl/private:/keys" - "/home/docker-distro/registry/auth:/auth" Hint Keep security in mind!
- 16. Introduction The pipeline The environment The images The problems Finish The CI Server TeamCity Free version provides: 3 build agents 20 build configurations Broad configuration capabilities (build parameters, configuration templates, defining dependencies etc)
- 17. Introduction The pipeline The environment The images The problems Finish Understanding CI Build Server and Build Agents
- 18. Introduction The pipeline The environment The images The problems Finish Customs Special forces
- 19. Introduction The pipeline The environment The images The problems Finish Customs Custom Agent FROM sjoerdmulder/teamcity-agent:latest MAINTAINER Grzegorz Godlewski <[email protected]> RUN apt-get -y install software-properties-common RUN apt-add-repository ppa:ansible/ansible RUN apt-get update RUN apt-get -y install ansible COPY ./keys/id_rsa /home/teamcity/.ssh/id_rsa RUN chown -R teamcity:teamcity /home/teamcity/.ssh ADD docker-entrypoint.sh /docker-entrypoint.sh RUN chmod a+x docker-entrypoint.sh
- 20. Introduction The pipeline The environment The images The problems Finish Customs Extending the images
- 21. Introduction The pipeline The environment The images The problems Finish Implementation TeamCity Server server: image: "sjoerdmulder/teamcity:latest" ports: - "8111:8111" volumes: - "/home/teamcity/server/data:/var/lib/teamcity" environment: TERM: linux
- 22. Introduction The pipeline The environment The images The problems Finish Implementation TeamCity Agent agent: image: "my-own-hub/teamcity-agent:latest" ports: - "9090:9090" volumes: - "/var/run/docker.sock:/var/run/docker.sock" # docker in docker - "/usr/bin/docker:/usr/bin/docker" - "/usr/bin/docker:/usr/local/bin/docker" - "/home/teamcity/agent/work:/opt/buildAgent/work" # work directories - "/home/teamcity/agent/composer:/opt/composer/cache" # composer cache - "/home/teamcity/agent/docker:/home/teamcity/.docker" # hub auth keys environment: TERM: linux TEAMCITY_SERVER: "http://teamcity_server:8111" # your server TEAMCITY_AGENT_NAME: "Alpha" # readable name AGENT_HOME_ON_HOST: "/home/teamcity/agent" # custom ENV for runtime
- 23. Introduction The pipeline The environment The images The problems Finish Implementation The flow 1 Run build in dev image 2 Package using dist image 3 Distribute using Ansible and docker-compose
- 24. Introduction The pipeline The environment The images The problems Finish It’s never that easy Not so easy... Cleaning up CI process performance Proper kernel modules Dynamically linked modules Race conditions
- 25. Introduction The pipeline The environment The images The problems Finish It’s never that easy https://github.com/docker/docker/issues/4036
- 26. Introduction The pipeline The environment The images The problems Finish It’s never that easy https://github.com/docker/docker/issues/4036 If you’re using the devicemapper diver, make sure that Udev Sync Supported is set to true.
- 27. Introduction The pipeline The environment The images The problems Finish Questions?
- 28. Introduction The pipeline The environment The images The problems Finish Thank you! http://linkedin.com/in/ggodlewski [email protected] @GGodlewski