Download as PDF, PPTX
Uploaded byThe Linux Foundation
Rootlinux17: An introduction to Xen Project Virtualisation
The document outlines the developments and capabilities of the Xen Project, including its community contributions, security features, and virtualization capabilities. Key topics include live patching for security updates, virtual machine introspection for malware protection, and various use cases and configurations in virtualization. It emphasizes strong isolation, flexibility, and collaboration within the community for enhanced performance and security in cloud computing environments.
More Related Content
Similar to Rootlinux17: An introduction to Xen Project Virtualisation
![XPDDS19: [ARM] OP-TEE Mediator in Xen - Volodymyr Babchuk, EPAM Systems](https://cdn.slidesharecdn.com/ss_thumbnails/xendevsummit2019-babchuk-op-tee-190812095541-thumbnail.jpg?width=640&height=640&fit=bounds)
![Governance, Deployment & Methodologies for Agentic Automation [2/3]](https://cdn.slidesharecdn.com/ss_thumbnails/agenticcommunityseries-day2-251112135038-a386476d-thumbnail.jpg?width=640&height=640&fit=bounds)
Rootlinux17: An introduction to Xen Project Virtualisation
- 1. Lars Kurth Community Manager,Xen Project Chairman, Xen Project Advisory Board Director, Open Source, Citrix lars_kurth www.slideshare.net/xen_com_mgr/presentations
- 2. Was a contributorto various projects Worked in parallel computing, tools, mobile and now virtualization Community guy for the Xen Project Working for Citrix Accountable to the Xen Project Community Chairman of Xen Project Advisory Board
- 4. Consolidation Servers/Equipment, Cooling, Floorspace Faster provisioning Flexibility Less dependency on specific Hardware Co-existing OS environments Increased uptime Live migration, storage migration, fault tolerance, HA Enhanced security
- 5. Strong Isolation Architecture providesstrong isolation between VMs Grant tables System Partitioning Disaggregation: sandboxing parts of the system Fine-grain control of VM capabilities Enables multi-layered security approach Secure I/O Sandboxing disk, memory, etc. drivers New classes of threat detection Virtual Machine Introspection, alt2pm
- 6. Consolidation Single SoC Maintainability, BoM Flexibility Lessdependency on specific Hardware Co-existing OS environments Additional Requirements Security requirements (same as on previous slide) Minimal IRQ latency Safety Certification Low or 0 scheduling overhead Drivers for special I/O devices
- 7.
- 8.
- 9. Introduction of keyconcepts
- 10. HWCPUsMemoryI/O VM1 (DomU1) Guest OS Applications VM0(or Dom0) Dom0 Kernel Native Driver VM2 VMn Applications Guest OS Applications Guest OS Toolstack Scheduler MMU Timers InterruptsConfig *Back Driver *Front Driver
- 11. Marketing Term ModeWith HVM / Fully Virtualized HVM HVM + PV drivers 1 HVM PV Drivers PVH PV pvh=1 PV PV Qemu Qemu HW PV Qemu HW HWPV PV PV PV PV WindowsLinux,BSDs,… ARM N/A PV PV HW 1) When running a Linux/BSD guest on HVM with PV drivers, we also use the term PVHVM User perspective (config file)
- 12. HW P HW PVH PV PVPV PV PV PV PV ARM N/A PV PV Simplicity: Less code & fewer Interfaces in Linux/FreeBSD Security : smaller TCB and attack surface, fewer possible exploits Clean-up : simplify Xen-Linux kernel, Xen-Any-OS interface Better Performance & Lower Latency for some workloads Dom0 must be a PV guest: run Dom0 using hardware extensions 32 bit: PV guest kernels were run in ring 1, userspace in ring 3 (HW isolation) 64 bit: no ring 1 & 2 kernel & user space must share ring 3 (TLB flushes) This is the most complex part of Xen today!
- 13.
- 14. A tale ofimproved collaboration within the Xen Project Community
- 15. 2015 2016 2017 4.84.7P Why did we develop Live Patching? Affected AWS, Rackspace, IBM SoftLayer and many others Deploying security patches may require reboots; Inconveniences users How did we fix this? 2015: Design with input from AWS, Alibaba, Citrix, Oracle and SUSE Replace functions while running (old with new) in a payload Stackable payloads can be applied and removed 2016: Xen 4.7 came with Live Patching for x86 2016: Xen 4.8 added extra x86 use-cases and ARM support 2017: XenServer 7.1 releases Live Patching in first commercial product … D
- 16. Pratap Sankar @Flickr Specification & Status xenbits.xen.org/docs/unstable/misc/livepatch.html wiki.xenproject.org/wiki/LivePatch Presentations, Videos, Demos bit.do/live-patch-detailed-ppt bit.do/live-patch-detailed-video bit.do/live-patch-short-ppt bit.do/live-patch-short-video
- 17. A new wayto protect against malware
- 18. 2007 2009 20112013 2015 2017 Enablers: from xenaccess/xenprobes to LibVMI Interesting research topic Originally used for forensics (too intrusive for server virt) AA A Xenaccess/Xenprobes LibVMI A Products AIS Introvirt, BitDefender Hypervisor Introspection, Zentific Zazen VMI: enabling commercial applications Hardware assisted VMI solves the intrusion problem Collaboration between: Zentific, Citrix, BitDefender, Intel and others VMI: HW Support (EPT, …) ARM, alt2pm, .. C CC
- 19. VM3 Guest OS App VMn Guest OS App VM2 GuestOS App Dom0 Dom0 Kernel Drivers Agent(s) Agent(s) Agent(s) Installed in-guest agents, e.g. anti-virus software, VM disk & memory scanner, network monitor, etc. Can be disabled by rootkits and advanced persistent threats (APT)
- 20. Several VM3 VMnVM2Dom0 Dom0 Kernel Drivers VM3 GuestOS App VMn Guest OS App VM2 Guest OS App Security Appliance VM1 Introspection Engine Protected area XSM/Flask or another mechanism to protect the IF Uses HW extensions to monitor memory (e.g. Intel EPT) Low Intrusion Register rules with Xen to trap on and inspect suspicious activities (e.g. execution of memory on the dynamic heap)
- 21. All malware needan attack technique to gain a foothold Attack techniques exploit specific software bugs/vulnerability The number of available attack techniques is small Buffer Overflows, Heap Sprays, Code Injection, API Hooking, … Because VMI protects against attack techniques It can protect against entirely new malware Verified to block these advanced attacks in real-time APT28, Energetic Bear, DarkHotel, Epic Turla, Regin, ZeuS, Dyreza, … solely by relying on VMI
- 22. Rootkits & APTs Exploit0-days in Operating Systems/System Software Can disable agent based security solutions (mask their own existence) VMI solutions operate from outside the VM Thus, it cannot be disabled using traditional attack vectors BUT: VMI is not a replacement, for traditional security solutions It is an extra tool that can be used to increase protection
- 23. Pratap Sankar @Flickr Documentation wiki.xenproject.org/wiki/Virtual_Machine_Introspection Products AIS Introvirt XenServer www.ainfosec.com Zentific Zazen (Apr 17) Xen & XenServer & … www.zentific.com Protection & Remedial Monitoring & Admin Forensics & Data gathering Malware analysis BitDefender HVI XenServer www.bitdefender.com Protection & Remedial Monitoring & Admin
- 24. Pratap Sankar @Flickr How secure is the Xen Project Hypervisor really?
- 25. Vulnerability data fromcvdetails.com 0 50 100 150 200 250 2016 2015 2014 2013 2012 All CVE’s 2015+ Active initiatives to find bugs XTF to help find bugs Fuzzing of some components Very few ARM issues 2016: 2/33 2015: 6/47 Does not use QEMU Xen Linux Kernel QEMU
- 26. 0 200 400 600 800 1000 0 97 4 CVE’s by CVSSSeverity Average CSSV Scores Xen: 4.7 Linux Kernel: 5.9 QEMU: 4.3 Known 0-Day Exploits Xen: 0 Linux Kernel: 18 QEMU: 0 Low = 0.1-3.9; Medium = 4.0-6.9; High = 7.0-8.9; Critical = 9.0-10.0 Xen Linux Kernel QEMU
- 27. Team Process Severity1 Type CVEs Days 2 Who? 3 Xen Hypervisor Includes Linux & QEMU vulnerabilities in supported Xen configurations Yes Yes All Responsible Yes 14 D, S, P OpenStack OSSA OpenStack OSSN Yes Yes Yes Yes > Low <= Low Responsible Full, post-fix Yes 3-5 D, S, P Linux Kernel via OSS security distros OSS security Yes Yes Partly 4 Yes No > Low <= Low Responsible Full, immediate Yes 14-19 D 4 QEMU via OSS security distros OSS security Yes Partly 4 > Low <= Low Responsible 5 Full, immediate Yes 14-19 D 4 Jailhouse No No 3) D = Distros/Products, S = Public Service, P = Private Downstream 4) No own pre-disclosure list 5) Only handles x86 KVM bugs 1) Is the CVE severity used as cut-off for the process? 2) Days embargoed Responsible only
- 28. HWCPUsMemoryI/O Dom0 Dom0 Kernel VM1 uKernel xtf.git arch xtf.gitcommon Each test is a file in xtf.git tests test_main() In essence a unikernel per test, with fewer safeguards in place to allow for easy testing of corner cases Also used for Vulnerability Investigation and Testing Qemu Xenstore *Back Drivers hypercalls evtchn gnttab x86 emulator
- 29. Picture by LarsKurth
- 30. Track Record 81% ofVulnerabilities Low and Medium Average severity of vulnerabilities getting lower Hardening Activities Security Audits by Cloud and Product Vendors Testing (fuzzing, XTF, code inspection, …) Industry Leading Vulnerability Process Includes QEMU and Kernel XSAs Designed with input from Cloud Providers Isolation Limits impact of exploits Picture by Lars Kurth
- 32. 2007 2009 20112013 2015 2017 A A A Q Q C C C CC Technology enablers: XSM, vTPM & TXT (TEE), Disaggregation & Driver Domains Qubes OS Architecture, Qubes OS 1.0, … 2009: Project Independence (Intel / Citrix) 2010: XenClient 1.0 2013: XenClient XT 2014: Became OpenXT (BAE Systems, Assured Information Security) 2015: Support for Cell Phones, Tablets and Embedded Devices Crucible:Defense uXen (Bromium) – Windows only, thus never made it upstream Du
- 33. Dom0 Toolstack HWI/O DomU Guest OS Applications Dom0 Kernel NativeDriver *Back Driver *Front Driver Config
- 34. DomU Guest OS Applications Dom0 Dom0 Kernel NetFrontDriver BlockFront Driver Toolstack Config Driver Domain Guest OS*: Linux, BSD, MiniOS, unikernel, … Disk Controller Storage Domain Guest OS* Disk Driver BlockBack Driver Network Domain Guest OS* Network Driver NetBack Driver Netw Controller Qemu Xenstore
- 35. Attack Surface Reduction Similarto Linux Security Modules/SELinux Same policy syntax as SELinux Different types, roles, users and attributes Same tools for policy compilation / verification (checkpolicy) VM hypervisor domain(self) domain(other) memory (grant, mmu, shadow) inter-VM communicationpassthroughsecurity config Fine-grained policy, controlling which hypervisor functionality is accessible to this (class of) VM Effect: limit what an exploit in this VM could do
- 37. User defined AppVMs for individual apps or groups of apps USB Service Domain Banking Domain Personal Domain Firewall VM enforces network policies Network Domain Dom0 Secure UI and sysadmin domain
- 38. User defined AppVMs for individual apps or groups of apps USB Service Domain Banking Domain Personal Domain Firewall VM enforces network policies Network Domain Dom0 Secure UI and sysadmin domain Tor VM pre-configured from whonix.org
- 39. Pratap Sankar @Flickr Crucible:Defense starlab.io Xen Project based virtualization platform for technology protection, cyber-hardening, and system integrity for aerospace & defense systems Documentation wiki.xenproject.org/wiki/Dom0_Disaggregation wiki.xenproject.org/wiki/Xen_Security_Modules_:_XSM-FLASK Products & Projects Qubes OS www.qubes-os.org Secure OS OpenXT www.openxt.org FOSS Platform for security research, security application and embedded appliance integration building on Xen & OpenEmbedded
- 41. Pratap Sankar @Flickr AIS ainfosec.com BAE Systems baesystems.com Galois galois.com Maintain FreeRTOS Xen Port Developed and maintain HalVM Dornerworks dornerworks.com/xen Consulting Xen Embedded Distros Xen for Xilinx Zynq Xen for NXP i.MX 8 ARLX Hypervisor DO-178 (EAL6+), IEC 62304, ISO 26262 MILS EAL FACE, VICTORY, ARINC 653 Starlab starlab.io Crucible and Crucible:Defense Xen embedded hypervisor In progress: DO-178, MILS EAL Uses a minimal Dom0 using MiniOS, disaggregation and XSM/FLASK Precedents of military grade certification for Xen based systems www.slideshare.net/xen_com_mgr/art-certification & www.youtube.com/watch?v=UyW5ul_1ct0 www.linux.com/news/xen-project/2017/2/how-shrink-attack-surfaces-hypervisor
- 42. Additional Requirements Security requirements Safetycertification Low or 0 scheduling overhead Minimal IRQ latency Drivers for special I/O devices
- 44. Xen supports severaldifferent schedulers with different properties.
- 45. Xen supports severaldifferent schedulers with different properties. Regular VM scheduler (Credit) Hard real-time (ARINC653) Dedicated to 1 VCPU (pinning) no scheduler overheads Soft real-time (RTDS)
- 46. Scheduler Use-cases Xen4.8 Future plans Credit General Purpose Supported Default Supported Optional Credit 2 General Purpose Optimized for lower latency, higher VM density Supported Default RTDS Soft & Firm Real-time Multicore Embedded, Automotive, Graphics & Gaming in the Cloud, Low Latency Workloads Experimental Better XL support <1μs granularity Supported Hardening Optimization ARINC 653 Hard Real-time Single core Avionics, Drones, Medical Supported Compile time Legend: Likely in 4.9 or 4.10 Possible in 4.10
- 47. A53 Programmable Logic (FPGA) R5R5A53A53 A53 Dom0 Dom0 Kernel Toolstack Bare Metal1 Bare Metal ELF payload FPGA Library VM1 - VMn DomU Kernel Application FPGA Driver Dedicated Bare Metal2 Bare Metal ELF payload Dedicated dornerworks.com/wp-content/uploads/2017/01/Xen-Zynq-Distribution-XZD-Users-Manual.pdf
- 48. vCPU 0 pCPU 0 vCPU1 pCPU 1 irq 109 virq 109 IRQ injection Always on the CPU running the vCPU
- 49. vCPU 0 pCPU 0 irq109 virq 109 vCPU 1 pCPU 1 IF vIRQ target changes or vCPU is moved THEN vIRQ is moved immediately virq 109
- 50. vCPU 0 pCPU 0 vCPU1 pCPU 1 irq 109 virq 109 IRQs always shadow the vIRQ minimizes latency Xilinx ZynqMP board (four Cortex A53 cores, GICv2) WARM_MAX (excluding the first 3 interrupts): <2000ns marc.info/?l=xen-devel&m=148778423725945 marc.info/?l=xen-devel&m=148839743820338
- 51. DomU Xen DomU Xen irq 109 virq 109 GIGC_LH Write EOI DomU Xen Maintenance interrupt GIGC_LH Clear IRQreceived by DomU DomU performs EOI The guest kernel issues an "EOI” at the end of the interrupt service routine, to notify the HW that the IRQ handling is finished. No maintenance IRQ Additional context switch to handle EOI. Use EOI support in HW to directly EOI the physical IRQ
- 53. Existing net, block, console keyboard,mouse, USB framebuffer, GPU sharing* New 9pfs PVcalls multitouch, sound, display, DRM Developing New Ones Easy to write (GPL and BSD samples) Kernel and User Space *) A number of different approaches by different vendors in different market segments are being deployed, which are PV-like, but not strictly a PV protocol
- 54. Vehicles are becomingthe ultimate mobile device
- 55. Pratap Sankar @Flickr LG Electronics Demo bit.do/lg-xen-demo-2016 Bosch Car GmbH Contributions 10 smaller features in 2016 Perseus Founded by Xen maintainer bit.do/perseus-2017 GlobalLogic Product: Nautilus bit.do/gl-nautilus First product in production expected in Q1 2018 Supports: HW: Renesas R-Car Gen2 & Gen3, TI Jacinto6, Intel Apollo Lake, Qualcomm 410C, Sinlinx A33 Guests: Linux up to 4.9 Android M, N, N-Car QNX, ThreadX, FreeRTOS PV Drivers for: GPU, Audio, HW accelerated Video codecs, DRM, … Contributions: 27 smaller features from 2013 to 2016 EPAM Demo youtube.com/watch?v=jMmz1odBZb8 Interesting Features: Container based telematics applications running in a Xen VM that can be downloaded from a cloud service Ongoing Contributions: ABIs for PV Sound, PV Display & PV DRM Leading development of co-processor sharing framework
- 56. Dom0 System Control DomU Cluster Domain AutomotiveGrade Linux or QNX DomU HMI Domain HMI Realtime OS CAN Processing HW Drivers Hypervisor Application Core Realtime Core Watchdog PV back* Drivers HW Drivers Linux Kernel HW DriversPV front* Drivers MW Frameworks RVC, SVS Instrument Cluster Automotive Grade Android HW DriversPV front* Drivers Connectivity Media AUTOSAR / RTOS Tell Tales RVC = Rear View Camera SVS = Service Vehicle Soon HMI = Human Machine Interface Tell Tales = various status indicators
- 57. AWS Dom0 - ControlDomD – HW Drivers & Cluster Wayland/Weston OpenGL ES Linux Kernel with GPU and other HW Drivers ALSA w PV_ALSAS_BE DomU Fusion Container mgmt tool Linux Kernel w/o HW Drivers Minimal rootfs with systems library Telematics simulation Agent (Acceleration, Braking, Corning, GPS) DomU – Linux IVI MW Frameworks PV DISPLAY Linux Kernel with GPU and w/o other HW Drivers PV EVENTS PV SOUND IVI Simulation App Trusted Apps TrustZone Hypervisor R-Car H3 Platform OP-TEE OS TZ monitor Driver Behavior Based Insurance Backend Telematics Simulation Agent ver 2.0 Telematics Simulation Agent ver 1.0 Monitoring Dashboard Wayland BE (Events/Display) Cluster Simulation AppDom0 Services Minimal rootfs Linux Kernel w/o HW Drivers Containers
- 59. 0 500 1000 1500 2000 2500 3000 3500 2011 2012 20132014 2015 2016 Stats are impacted by release model (code freezes) and our transition to 2 releases per year 4.7 4.84.1 4.2 4.3 4.4 4.5 4.6
- 60. Top: Citrix 48% Suse 17% Oracle6% Intel 6% Red Hat 4% Linaro 3% FreeBSD 2% Star Lab 1% Other 13% Others: Fujitsu Invisible Things Lab BitDefender Huawei Zentific Verizon Cavium GlobalLogic NSA …
- 61. Top: Citrix 39% Suse 22% Oracle7% ARM 5% Red Hat 4% Linaro 3% Intel 3% Star Lab 2% BSD 2% Fujitsu 2% Bitdefender 2% Zentific 1% NSA 1% Zentific 1% Qualcomm 1% Huawei 1% Other 6% First-time contributors in 2016: ARM Aporeto Bosch Car Multimedia Gmbh Netflix Qualcomm Xilinx
- 62. Picture by LarsKurth
- 63. Extremely Flexible andVersatile Proven in different markets Easy to port to new environments Easy to develop new PV drivers Highly customizable Security and Resilience Isolation, Partitioning, Security Features Track record in handling security issues Safety Examples of Military Grade Certification Vibrant and Diverse Community Covering Server, Cloud, Security, Embedded, Automotive Picture by Lars Kurth
- 64. Developer Portal: bit.do/xen-devs Xenon ARM whitepaper: bit.do/xenarm-white Xen on ARM wiki: bit.do/xenarm-wiki Port Xen to a new SOC: bit.do/xenarm-porting Add Xen support Xen to your OS: bit.do/xenarm-os Device Passthrough presentation: bit.do/xenarm-pt OE meta-virtualization Xen recipe: bit.do/xenmeta OpenXT (Xen + OpenEmbedded): openxt.org Xenbedded presentation: bit.do/xenbedded Monthly ARM Community Call: bit.do/xenarm-call
- 65. Lists and IRCon freenode: [email protected] [email protected] #xenarm or #xen-devel Internships in 2017: Google Summer of Code (Students) Outreachy (Women and other groups) bit.do/xen-internships Xen Project Developer and Design Summit: July 11-13, Budapest, Hungary CfP: open until April 14 bit.do/xensummit17
- 66. Picture by LarsKurth