Mastering Proxmox

Table of Contents

Mastering Proxmox

Credits

About the Author

About the Reviewers

www.PacktPub.com

Support files, eBooks, discount offers, and more

Why subscribe?

Free access for Packt account holders

Preface

What this book covers

What you need for this book

Who this book is for

Conventions

Reader feedback

Customer support

Downloading the example code

Errata

Piracy

Questions

1. Dive into the Virtual World with Proxmox

Proxmox cluster required

The Proxmox Graphical User Interface (GUI)

The GUI menu system

Menu chart

The Datacenter menu

The Search tab

The Storage tab

The Backup tab

Node-specific tabs

The Summary tab

The Network tab

The Syslog tab

The UBC tab

The Subscription tab

The Updates tab

The Ceph tab

Virtual machine tabs

The Summary tab

The Hardware tab

The Options tab

The Backup tab

The Snapshots tab

The Permissions tab

Setting up a basic cluster

The hardware list

The software list

Hardware setup

Proxmox installation

Cluster creation

Proxmox subscription

Attaching shared storage

Adding virtual machines

Main virtual machine

Creating a KVM virtual machine

Creating an OpenVZ virtual machine

Proxmox cloning/template

Introducing cloning using a template

Transforming VM into a template

Cloning using a template

Full Clone versus Linked Clone

VM migration

Summary

2. Proxmox Under the Hood

The Proxmox cluster directory structure

Dissecting the configuration files

The cluster configuration file

The storage configuration file

Local directory-based storage

NFS-shared storage

iSCSI/LVM shared storage

User configuration files

The password configuration file

The virtual machine configuration file

Arguments in the KVM configuration file

The Proxmox OpenVZ configuration file

The version configuration file

Member nodes

The .members file

The virtual machine list file

The cluster logfile

Summary

3. Shared Storages with Proxmox

Local storage versus shared storage

Live migration of a virtual machine

Seamless expansion of multinode storage space

Centralized backup

Multilevel data tiering

Central storage management

Local and shared storage comparison

Virtual disk image

Supported image formats

The .qcow2 image type

The .raw image type

The .vmdk image type

Image file manipulation

Resizing virtual disk image

Moving a virtual disk image

Storage types in Proxmox

Directory

Logical Volume Management

Network File System

RADOS Block Device

GlusterFS

Noncommercial/commercial storage options

FreeNAS – budget shared storage

Summary

4. A Virtual Machine for a Virtual World

Creating a VM from a template

Advanced configuration options for a VM

The hotplugging option for a VM

The hotplugging option for .conf

Loading modules

Adding virtual disk/vNIC

Nested virtual environment

Enabling KVM hardware virtualization

Network virtualization

Backing up a virtual machine

Proxmox backup and snapshot options

Backing up a VM with a full backup

Creating a schedule for Backup

Node

Storage

Day of Week

Start Time

Selection mode

Send email to

Compression

Mode

Creating snapshots

Deleting old backups

Restoring a virtual machine

Command-line vzdump

Backup configuration file – vzdump.conf

#bwlimit

#lockwait

#stopwait

#script

#exclude-path

Summary

5. Network of Virtual Networks

Introduction to a virtual network

Physical network versus virtual network

Physical network

Virtual network

Networking components in Proxmox

Virtual Network Interface Card (vNIC)

Virtual bridge

Virtual LAN (VLAN)

Network Address Translation/Translator (NAT)

Network bonding

Components naming convention

Network configuration file

bridge_stp

bridge_fd

Adding a virtual bridge

Adding a bonding interface

Adding NAT/masquerading

Adding VLAN

Sample virtual networks

Network #1 – Proxmox in its simplest form

Network #2 – multitenant environment

Network #3 – academic institution

Multitenant virtual environment

Multitenant network diagram

Summary

6. Proxmox HA – Zero Downtime

Understanding High Availability

High Availability in Proxmox

Requirements for HA setup

Fencing

Configuring Proxmox HA

Setting up node BIOS

Creating an APC-managed PDU user

Configuring Proxmox fencing

Configuring virtual machine HA

Testing Proxmox HA

Fencing manually

Proxmox HA need to know

Summary

7. High Availability Storage for High Availability Cluster

Introducing the Ceph storage

Object Storage

Block Storage

Filesystem

Reasons to use Ceph

Virtual Ceph for training

The Ceph components

Physical node

Maps

Cluster map

CRUSH map

Monitor

OSD

OSD Journal

MDS

Placement Group (PG)

Pool

Ceph components summary

The Ceph cluster

Hardware requirements

Software requirements

Installing Ceph using an OS

Installing and setting up Ubuntu

Creating an admin user

Assigning SUDO permission to a user

Updating Ubuntu

Generating an SSH Key

Installing ceph-deploy

Creating a Ceph cluster

Installing Ceph on nodes

Creating Monitors (MONs)

Gathering the admin keys

Creating OSDs

Connecting Proxmox to a Ceph cluster

Installing Ceph on Proxmox

Preparing a Proxmox node for Ceph

Installing Ceph

Creating MON from the Proxmox GUI

Creating OSD from the Proxmox GUI

Creating a new Ceph pool using the Proxmox GUI

Creating a Ceph FS

Setting up an MDS daemon

Setting up Ceph FS using FUSE

Mounting Ceph FS

Connecting Proxmox to Ceph FS

Learning Ceph's CRUSH map

Extracting the CRUSH map

Decompiling the CRUSH map

Editing the CRUSH map

Compiling the CRUSH map

Injecting the CRUSH map into the cluster

Verifying the new CRUSH map

Managing Ceph pools

Creating a new Ceph pool using the CLI

Verifying the new Ceph pool

Adding OSDs to a pool

Assigning a pool to the ruleset

Connecting Proxmox to the new pool

Ceph benchmarking

The Ceph command list

Summary

8. Proxmox Production Level Setup

Defining a production level

Key parameters

Stable and scalable hardware

Current load versus future growth

Budget

Simplicity

Tracking the hardware inventory

Hardware selection

An entry-level Proxmox production setup

An i7-based Proxmox node

A Xeon-based Proxmox node

An entry-level Ceph production setup

An advanced-level Proxmox production setup

A Xeon-based Proxmox node

An advanced-level Ceph production setup

Desktop class versus server class

Brand servers

Hardware tracking

AMD-based hardware selection

An AMD-based entry-level Proxmox

An AMD-based advanced-level Proxmox

An AMD-based Ceph setup

Performance comparison

Summary

9. Proxmox Troubleshooting

Main cluster issues

GUI shows everything is offline

Rejoining a Proxmox node with the same IP address

Disabling fencing temporarily

The occurrence of kernel panic when disconnecting USB devices

The occurrence of VM shutdown error when initiated from GUI

Kernel panic on Proxmox 3.2 with HP NC360T

VMs not booting after you restart the network service

Proxmox cluster is out of Quorum and cluster filesystem is read only

Proxmox boot failure due to the getpwnam error

Cannot log in to GUI as ROOT

Booting with a USB stick fails in Proxmox

The Upgrade from Proxmox 3.1 to Proxmox 3.2 is disabled through GUI

VZ kernel 2.6.32-28-pve breaks libnl/netlink in host and VM

Nodes not visible on the Proxmox GUI after an upgrade

GRUB is in an endless loop after Proxmox installation

SSH access is possible but Proxmox node does not reboot

Storage issues

Deleting damaged LVM with error read failed from 0 to 4096

Proxmox cannot mount NFS share due to time-out error

Removing stale NFS shares when a stale file handle error occurs

The occurrence of '--mode session exit code 21' errors while accessing iSCSI target

Cannot read an iSCSI target even after it has been deleted from Proxmox storage

OSDs still show up in Proxmox after you remove the Ceph node

The 'No Such Block Device' error that shows up during creation of an OSD

The fstrim command does not trim unused blocks for Ceph

The 'RBD Couldn't Connect To Cluster (500)' error when connecting Ceph with Proxmox

Changing the storage type from ide to virtio

The 'pveceph configuration not initialized (500)' error for the Ceph tab

Ceph FS storage disappears after a Proxmox node reboots

VM cloning does not parse in Ceph storage

Network connectivity issues

No connectivity on Realtek RTL8111/8411 Rev. 06 NIC

Network performance is slower with e1000 vNIC

KVM virtual machine issues

Windows 7/XP machine converted to Proxmox KVM hangs during boot

Windows 7 VM only boots when rebooted manually

The Proxmox 3.2 upgrade adds two com ports and one parallel port to the Windows VM

The qemu-img command does not convert the .vmdk image files created with the .ova template in Proxmox VE 3.2

Online migration of a virtual machine fails with a 'Failed to sync data' error

Change in memory allocation is not initialized after a VM is rebooted

The virtio virtual disk is not available during the Windows Server installation

OpenVZ container issues

The creation of OpenVZ container takes a long time on NFS or GlusterFS storage

OpenVZ containers are no longer shown after a cluster is created

Header error during the installation of PF_RING in Proxmox

Backup/restore issues

A Proxmox VM is locked after backup crashes unexpectedly

Backing up only the primary OS virtual disk

Backup of VMs stops prematurely with an 'Operation Not Permitted' error

A backup task takes a very long time to complete, or it crashes when multiple nodes are backing up to the same backup storage

Backup of virtual machines aborts a backup task prematurely

Backup storage has a lot of .dat files and .tmp directories using the storage space

VNC/SPICE console issues

The mouse pointer is not shared with SPICE-VIEWER on Windows 8 VM

The SPICE console has become unstable after the Proxmox VE 3.2 update

Remote Viewer is unable to connect to a SPICE-enabled virtual machine on Windows OS

Summary

10. Putting It All Together

Scenario #1 – academic institution

Scenario #2 – multitier storage cluster using Proxmox cluster

Scenario #3 – virtual infrastructure for multitenant cloud service provider

Scenario #4 – a nested virtual environment for a software development company

Scenario #5 – a virtual infrastructure for the public library

Scenario #6 – multifloor office virtual infrastructure with virtual desktops

Scenario #7 – virtual infrastructure for hotel industry

Scenario #8 – virtual infrastructure for a geological survey organization

Network diagrams for scenarios

Summary

Index

Mastering Proxmox

Mastering Proxmox

Copyright © 2014 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing, and its dealers and distributors will be held liable for any damages caused or alleged to be caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

First published: July 2014

Production reference: 1070714

Published by Packt Publishing Ltd.

Livery Place

35 Livery Street

Birmingham B3 2PB, UK.

ISBN 978-1-78398-082-6

www.packtpub.com

Cover image by Duraid Fatouhi (<[email protected]>)

Credits

Author

Wasim Ahmed

Reviewers

Rocco Alfonzetti Jr.

Alessio Bravi

Oleg Butovich

Daniel Lench

Razique Mahroua

Commissioning Editor

Kartikey Pandey

Acquisition Editor

Mohammad Rizvi

Content Development Editor

Madhuja Chaudhari

Technical Editor

Rohit Kumar Singh

Copy Editors

Alisha Aranha

Sarang Chari

Mradula Hegde

Gladson Monteiro

Adithi Shetty

Project Coordinator

Neha Bhatnagar

Proofreaders

Simran Bhogal

Amy Johnson

Linda Morris

Indexers

Mehreen Deshmukh

Rekha Nair

Tejal Soni

Graphics

Ronak Dhruv

Production Coordinator

Komal Ramchandani

Cover Work

Komal Ramchandani

About the Author

Wasim Ahmed, born in Bangladesh and now a citizen of Canada, is a veteran of the IT world. He was introduced to computers in the year 1992 and never looked back. Wasim has deep knowledge and understanding of network virtualization, big data storage, and network security. By profession, Wasim is the CEO of an IT support and cloud service provider company based in Calgary, Alberta. He serves many companies and organizations through his company on a daily basis. Wasim's strength comes from the experience he gained through learning and serving continually. Wasim strives on finding the most effective solution at the most competitive price point. He hand-built over a dozen enterprise production virtual infrastructures using Proxmox and Ceph storage system.

Wasim is notoriously known not to simply accept a technology based on its description alone, but put them through rigorous tests to check their validity. Any new technology that his company provides goes through months of continuous testing before it is accepted. Proxmox made the cut superbly.

I would like to thank all the staff at Proxmox for their support and dedication to the hypervisor community. I would also like to thank Packt Publishing for their vision of moving forward for this one-of-a-kind book on Proxmox and their support throughout the journey of making of this book.

About the Reviewers

Rocco Alfonzetti Jr. is an IT consultant for small businesses and has specialized in Linux and open source solutions for the last 15 years. Currently, he works for a software development company as an e-mail security expert. He lives in rural Connecticut with his wife and three children, and in his spare time, he enjoys beekeeping, raising chickens, and gardening.

Alessio Bravi has been playing with bits since he was five. He started programming at the age of six and soon focused his attention towards network administration and IT systems security in the best growing-up period of the Internet.

When he was 19, he founded IntSec.NET, and started working as CTO and Network and Security Administrator for Italian Internet service providers (ISPs/W-ISPs) and as an IT security consultant for many companies in Europe.

Alessio works only with Unix-like operating systems and is specialized in IT security analysis, network engineering and administration, autonomous systems BGP routing, IPv4 and IPv6 routing and switching, operating system virtualization, and data center management.

His personal blog can be found at http://blog.bravi.org/, where he writes some technical articles to share IT hints with the digital world. More technical skills and personal details about Alessio can be found on his LinkedIn© profile page at http://www.linkedin.com/in/alessiobravi.

Oleg Butovich is a freelance senior software developer with a passion for virtualization technologies. He has over 15 years of experience in the industry. He has worked on booking systems, trading platforms, laser image generators, digital media systems, medical and life science imaging systems, automatic inspection systems, and embedded systems.

Daniel Lench is a self-proclaimed fixer of all things. He is drawn to challenges, both physical and theoretical. His background includes acting as an artisan at a state museum, a production manager at a high-volume cabinet shop, AutoCAD expert for civil engineering firms and government agencies, and almost two decades of being professionally involved in the IT industry. In 2008, the challenge was to keep files in sync between multiple computers in real time. Since then he has been focused on finding the best answer. The NoFolder Project is an open source, real-time, private cloud-based backup, file synchronization, and collaboration service that is self hosted and administered in small business and enterprise settings. NoFolder addressed the policy and privacy concerns over using third-party services to store and share data. The project is for those concerned about data, the collaboration with it, and the preservation of it. The company maintains offices in the U.S. and the U.K. with additional resources in Sweden, Austria, and South Africa. Daniel is the founder as well as the CEO for NoFolder Ltd.

I would like to thank Rocco for introducing Proxmox to me. I would also like to thank Heather for the wonderful adventure.

Razique Mahroua is a technical consultant on High Availability systems as well as a technical writer. Currently involved in several open source projects, such as OpenStack and KVM, he has written about various technical topics for IBM and Amazon.

His experience ranges from cloud solutions, implementations (IaaS and PaaS), and by-products such as data clustering to network High Availability and data integrity. He currently assists several companies looking for best practices around cloud solutions.

www.PacktPub.com

Support files, eBooks, discount offers, and more

You might want to visit www.PacktPub.com for support files and downloads related to your book.

Did you know that Packt offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.PacktPub.com and as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at for more details.

At www.PacktPub.com, you can also read a collection of free technical articles, sign up for a range of free newsletters and receive exclusive discounts and offers on Packt books and eBooks.

http://PacktLib.PacktPub.com

Do you need instant solutions to your IT questions? PacktLib is Packt's online digital book library. Here, you can access, read and search across Packt's entire library of books.

Why subscribe?

Fully searchable across every book published by Packt

Copy and paste, print and bookmark content

On demand and accessible via web browser

Free access for Packt account holders

If you have an account with Packt at www.PacktPub.com, you can use this to access PacktLib today and view nine entirely free books. Simply use your login credentials for immediate access.

Preface

This book is well overdue in the world of virtualization. When I first came in contact with Proxmox several years ago, I did not have anything to fall back on other than Proxmox Wiki and forum. I learned Proxmox through lots of trial and error and very much had to reinvent wheels on my own in some cases. Since a lot of us went through the frustration and I personally do not feel others should have to invest a lot of time just to get to know Proxmox the hard way, this book has been written.

This book shows the inner workings of Proxmox including virtual network components, shared storage systems, nested virtualization, complex network topologies, and so on. With this book, we hope that the reader will be able to better equip themselves to face any virtualization challenges of any virtual infrastructure.

What this book covers

Chapter 1, Dive into the Virtual World with Proxmox, introduces Proxmox in general and the graphical user interface.

Chapter 2, Proxmox Under the Hood, introduces the Proxmox directory structure and configuration files.

Chapter 3, Shared Storages with Proxmox, explains how Proxmox interacts with the shared storage system and types of shared storage system supported.

Chapter 4, A Virtual Machine for a Virtual World, covers advanced virtual machine configurations such as enabling sound, USB devices, and so on.

Chapter 5, Network of Virtual Networks, explains the different networking components used in Proxmox to build virtual networks.

Chapter 6, Proxmox HA – Zero Downtime, explains the Proxmox High Availability (HA) feature and how to configure it.

Chapter 7, High Availability Storage for High Availability Cluster, explains a step-by-step process of setting up the Ceph cluster to be used as a shared storage system.

Chapter 8, Proxmox Production Level Setup, explains the type of hardware that should be and can be used in a production level Proxmox cluster setup.

Chapter 9, Proxmox Troubleshooting, lists real incidents with solutions that may arise in the Proxmox cluster.

Chapter 10, Putting It All Together, introduces several scenario-based virtual environments along with full network diagrams.

What you need for this book

Since we will be working with the Proxmox cluster throughout the book, it will be extremely helpful to have a working Proxmox cluster of your own. A very basic cluster of two Proxmox nodes and a storage node will do just fine.

Who this book is for

This book is for readers who want to build a virtual infrastructure purely based on Proxmox as hypervisor and Ceph as storage backend. Whether the reader is a veteran in the virtualized industry but has never worked with Proxmox, or somebody just starting out a promising career in this industry, this book will serve well.

Conventions

In this book, you will find a number of styles of text that distinguish between different kinds of information. Here are some examples of these styles, and an explanation of their meaning.

Code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles are shown as follows: The beginning of the tag shows the name of the cluster as name=pmx-cluster.

A block of code is set as follows:

1.0?>

pmx-cluster config_version=2>

/var/lib/pve-cluster/corosync.authkey>

  pmxvm01 votes=1 nodeid=1/>

  pmxvm02 votes=1 nodeid=2/>

Any command-line input or output is written as follows:

# ssh [email protected]

New terms and important words are shown in bold. Words that you see on the screen, in menus or dialog boxes for example, appear in the text like this: The Storage tab is probably one of the most important options in the Proxmox GUI.

Note

Warnings or important notes appear in a box like this.

Tip

Tips and tricks appear like this.

Reader feedback

Feedback from our readers is always welcome. Let us know what you think about this book—what you liked or may have disliked. Reader feedback is important for us to develop titles that you really get the most out of.

To send us general feedback, simply send an e-mail to <[email protected]>, and mention the book title via the subject of your message.

If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, see our author guide on www.packtpub.com/authors.

Customer support

Now that you are the proud owner of a Packt book, we have a number of things to help you to get the most from your purchase.

Downloading the example code

You can download the example code files