CloudStack EU user group - Trillian
Download as pptx, pdf1 like1,038 views
Trillian is an Ansible-based tool that automates the deployment of CloudStack environments for testing purposes. It supports building environments with various hypervisor and CloudStack versions across multiple concurrent tests. Jenkins integrates with Trillian to run test suites after package builds and clean up environments. The tools provide automated testing capabilities for CloudStack improvements and integrations.
More Related Content
What's hot (20)
Viewers also liked (20)
Similar to CloudStack EU user group - Trillian (20)
More from ShapeBlue (20)
Recently uploaded (20)
CloudStack EU user group - Trillian
- 1. The Cloud Specialists THE HITCH-HIKERS GUIDE TO CI/CD ShapeBlue.com • @ShapeBlue Paul Angus, VP Technology • @CloudyAngus
- 2. The Cloud Specialists ShapeBlue.com @ShapeBlue “ShapeBlue are expert builders of public & private clouds. They are the leading independent global CloudStack services company” A b o u t S h a p e B l u e
- 3. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue Paul Angus – VP Technology • Global authority on CloudStack & cloud infrastructure design. • 10+ years C-Level experience. • Apache CloudStack project committer • Specialising in deployment of CloudStack and surrounding infrastructure • USP, Georgian Ministry of Justice, Orange, TomTom, PaddyPower, Ascenty, BSkyB A b o u t M e
- 4. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue S h a p e B l u e c u s t o m e r s
- 5. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue S h a p e B l u e c u s t o m e r s
- 6. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue S h a p e B l u e c u s t o m e r s
- 7. The Cloud Specialists ShapeBlue.com @ShapeBlue Tr i l l i a n Why?
- 8. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue W h y “Building CloudStack environments using CloudStack and Ansible? Aren’t you just having fun?” CEO
- 9. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue W h y “Building CloudStack environments using CloudStack? Are you just having fun?” Talented Cloud Architect “If we can automate the building of environments including the networking, and generate any topology we want quickly, we can achieve the tasks that I’ve highlighted in these slides that I have with me, just in case you asked.”
- 10. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue W h y building of environments including the networking, and generate any topology we want quickly, we can achieve the tasks that I’ve highlighted in these slides that I have with me, just in case you asked.” CEO “That would be excellent. Go ahead. Oh, and here’s a pay rise”
- 11. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue We need to test: • New features • Hotfixes/patches • Full Versions • Physical integrations • New hypervisors • Upgrades • Against a broad range of deployment architectures • Against a broad range of hypervisor types and versions W h y
- 12. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue Many, many, MANY permutations of a CloudStack deployment…. • Basic / Advanced • Local / shared / mixed storage • More than 8 common hypervisor types/versions • 4 or 5 Management server OS possiblilities • That’s 144 combinations only looking the basics. • Previous attempts at test automation haven’t been able to get coverage or capacity. W h y
- 13. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue Discovery projects: • New Features • CloudStack integrations (ie Ceph) • CloudStack ecosystem components (ie ELK stack) W h y
- 14. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue Lab v1.0 • Templates to build nested hypervisor hosts • Cloudmonkey scripts to configure zone • Manual creation of storage mounts Simple but pretty effective H u m b l e B e g i n n i n g s
- 15. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue Lab v1.0 • A lot of small edits scripts to fit use cases. • Human error still a big factor • Can’t ‘set and forget’ • A lot of individual knowledge required • Loose definition of ‘testing’. H u m b l e B e g i n n i n g s
- 16. The Cloud Specialists ShapeBlue.com @ShapeBlue Trillian Tenets
- 17. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue Fully automated deployment of CloudStack environments from any CloudStack package repo Tr i l l i a n Te n e n t s
- 18. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue • Ability to support multiple concurrent environments • No hardcoding to a specific (our) lab/environment • As few separate components as possible • Separation of ‘code’ and ‘configuration’ • Scripted command-line use fully supported • Ability to support as many hypervisors as possible • As flexible as possible • Best ‘performance’ possible • Best utilization of hardware resources possible Tr i l l i a n Te n a n t s
- 19. The Cloud Specialists ShapeBlue.com @ShapeBlue Meet The Family
- 20. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue T h e C I F a m i l y Trillian Marvin Winston Jenkins Blue Orangutan
- 21. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue Trillian Ansible based environment builder • Multi-tenant (and multiple concurrent environments) • CloudStack builds the Virtual Infrastructure • Hypervisor hosts are CloudStack templates • Supports multiple and mixed hypervisors • Supports Advanced and Basic networks • Open source T h e C I F a m i l y
- 22. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue Trillian Installs & Configures: • CloudStack • MySQL • XenServer (configures) • vCenter (configures) • ESXi (configures) • KVM • Marvin hosts and Marvin test suites T h e C I F a m i l y
- 23. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue Marvin Python based test framework based on ‘nose’ test platform. Configuration files define environment to be tested. • Integral part of the CloudStack codebase • Recently given its own install package • Smoke and Component tests • 53 smoke test suites • 154 component test suites • Additional tests for plugins such as Nuage and SolidFire T h e C I F a m i l y
- 24. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue Jenkins CI Platform • Runs build jobs to produce RPMs from source code • Presents an integration point to wrap up the creation of environments using Trillian and the running of tests using Marvin. • Gives a UI to Trillian. T h e C I F a m i l y
- 25. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue Blue Orangutan Build-bot • Automation endpoint to send build jobs to Jenkins for CentOS 6/7 and Debian packages from CloudStack source • Provides an integrated way to create packages from pull requests in Github. • Can kick-off Jenkins jobs to build and test environments based on packages which it builds. T h e C I F a m i l y
- 26. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue Winston Test Orchestrator (Future project) • Receives ‘offers’ of test type/capacity from remote test labs • Takes request from Blue Orangutan to test packages and distributes tests to suitable remote labs. Offers might include: • Use of local storage • Use of basic vs advanced networks • Use of particular hypervisors/versions T h e C I F a m i l y
- 27. The Cloud Specialists ShapeBlue.com @ShapeBlue How its used
- 28. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue P r o c e s s u s e d i n A p a c h e C l o u d S t a c k P r o j e c t
- 29. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue • Test Pull Requests to add XenServer 7 support • XenServer won’t run on ESXi 5.5 • Create XenServer 7 VMs on stand-alone ESXi 6.0 host and ‘point’ Trillian at them. • vSphere 6.5 – Create vCenter and ESXi templates and run tests. X e n S e r v e r 7 & v S p h e r e 6 . 5
- 30. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue • Build base CloudStack environment quickly • Leverage CloudStack to create other infrastructure components just as quickly. ie • HostBill server • Cloudian manager server and storage nodes • Leverage CloudStack networking to give external access to environments – Static NAT, port forwarding or VPN. C l o u d i a n & H o s t B i l l
- 31. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue • Build required configuration with required components • Specific versions of CloudStack • Specific hypervisor types • Specific hypervisor versions • Manual creation of additional components still allowed • ie second zone with different hypervisor type or version S o l i d F i r e P l u g i n
- 32. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue We can do all of these things AT THE SAME TIME C o u p D e G r â c e
- 33. The Cloud Specialists ShapeBlue.com @ShapeBlue Trillian
- 34. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue https://github.com/shapeblue/Trillian Tr i l l i a n • Single zone • Use pre-set or custom Repos • Local or shared storage • ESXi, KVM (C6,C7,U14.04), XenServer hosts • Customisable number of primary and secondary storage pools • Multiple Management servers • Selectable Management server OS (C6.x, C7, U14.04) • Multiple hypervisors • dvSwitch (public/guest traffic) • Configurable Java versions • Option to use pre-built hosts • Multiple MySQL configurations
- 35. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue A n s i b l e i n Tr i l l i a n • All driven from Ansible • A global group_vars file stores all local configuration data such as: • Passwords • All defaults ie basic vs advanced or network naming • EVERYTHING can be customised at runtime. (except that) (and that)
- 36. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue ansible-playbook generate- cloudconfig.yml -i localhost --extra-vars "env_name='localstoragetest‘ env_version=cs49 mgmt=1 db=0 hvtype=x xs_ver=xs65sp1 hv=2 env_accounts=all pri=2 sec=1 build_marvin=yes mgmt_os=6 use_local_storage=true use_shared_storage_systemvms=true use_shared_storage=false" ansible-playbook deployvms.yml -i ./hosts_localstoragetest A n s i b l e i n Tr i l l i a n
- 37. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue Tr i l l i a n i n J e n k i n s • Different Jenkins’ jobs with specific ‘assumptions’. • Pull Requests receive URL from Blue Orangutan and automatically Run test suites, then clean up. • Reference_Trillian is a reference implementation of Jenkins and has more corner-case options exposed.
- 38. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue Tr i l l i a n i n J e n k i n s • Allows user to force a version of Java • Allows use of ‘pre-built’ hypervisors and hypervisors which cannot themselves be virtualised.
- 39. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue Tr i l l i a n i n J e n k i n s • If environment requires the hypervisor to use local storage, then CloudStack adds a data volume to hypervisor and configures hypervisor to use it for local storage. This keeps template size down • Mixed local and shared storage is supported
- 40. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue Internal ShapeBlue summary page UI plugin.
- 41. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue Tr i l l i a n & J e n k i n s i n C l o u d S t a c k • UI gives overall status/health of ‘Lab’ and environments • UI links to a job’s Jenkins output as well as UI of built environment and SSH session onto specific VMs
- 42. C l i c k t o e d i t The Cloud Specialists ShapeBlue.com @ShapeBlue I n S u m m a r y
- 43. The Cloud Specialists THE HITCH-HIKERS GUIDE TO CI/CD ShapeBlue.com • @ShapeBlue Paul Angus, VP Technology • @CloudyAngus