5. Creating and uploading virtual machine images

CSC provides a set of standard images that are well suited for cloud use. In most cases you can use these images instead of creating your own. These images are created automatically using a tool called diskimage-builder. If you are interested in the details about how these images are created, see this GitHub page. These images are updated at regular intervals so that they contain the latest security updates at the time virtual machines are launched.

It is possible that for some use cases the automatically created images are not suitable. In this case it is possible to create your own images and use them instead. However, there are some caveats that you need to consider when creating your own images that you would not need to consider when using the default images. This page documents some of these caveats and gives instructions on how to upload your own virtual machine images to cPouta.

5.1 Creating images

There are two different options for creating new virtual machine images: creating the image from scratch or launching a virtual machine based on an existing image, making modifications on the running machine and saving the changes as a new image by creating a snapshot.

5.1.1 Creating an image based on an existing image

Launch a virtual machine using one of the available images either through the Horizon web interface or through the command line interface. Here is the command for launching an instance from the command line:

openstack server create --flavor <flavor> \
--image <image uuid> \
--key-name <key name> \
--nic net-id=<name of network> \
--security-group default \
--security-group <additional security group> <name of server>

Login and make any necessary changes. Then create a snapshot of the machine's current state:

openstack server image create --name <name of snapshot to create> <name of vm>

It will take a little time to create the snapshot. Once it's finished, it will appear as a new image.

In the web UI under Compute | Instances, the instance specific Create Snapshot -menu items work for the same effect as the above CLI command. The snapshots created will appear in the Compute | Images section.

5.1.2 Creating an image from scratch

There are a number of tools for creating images from scratch. These tools can be categorized into tools that involve running an operating system install in a virtual machine for setting up the image and tools that take a base image and make modifications to it without running a virtual machine. We will call these "Installation based tools" and "Base image tools".

  Installation based tools Base image tools
Pros
  • Easy to use and understand
  • Familiar workflow for anyone familiar with manual OS installation
  • Fast
  • Possible to automate
  • Produces small images
  • Produces images suitable for clouds without modification
Cons
  • Slow
  • Upgrades are cumbersome
  • Care must be taken to make images cloud ready after initial installation
  • Images produced can be quite large
  • May be difficult to customize images
Examples virt-install, virt-manager, VirtualBox diskimage-builder, virt-builder

 

The generic workflow when using installation based tools is as follows:

  1. Obtain an installation media or a network installation link
  2. Start a virtual machine and point it to the installation media or network installation link
  3. Go through the installer
    • This step can optionally be automated using e.g. Kickstart
  4. After the installation is finished, shut down VM and use additional tools to prepare the image for cloud use

The generic workflow when using base image tools is as follows:

  1. Optionally customize configuration files that are used to generate the final image
  2. Determine suitable customization parameters
  3. Run a command to generate the final image

You can get more information about creating images in the very thorough "OpenStack virtual machine image guide". In particular, see chapters on creating images manually and tool support for creating images.

5.1.2.1 Caveats to keep in mind when creating images from scratch

These caveats usually only need to be considered when using installation based methods of image creation. The tools that use base images are usually specifically designed to create images for clouds, so they take care of these caveats for you. If you decide to use an installation based method of image creation, you should look into the excellent virt-sysprep tool that will take care of most modifications necessary for cloud use with a single command line command. This chapter lists some of the caveats that need to be handled before an image is ready for clouds.

cloud-init

There is a tool called cloud-init that must be installed on any images that are to be used in cPouta. It is used for certain tasks that need to be run when a virtual machine first boots up like generating SSH host keys and adding user SSH public keys.

User accounts (can be done with virt-sysprep)

Cloud images should only have a minimal set of user accounts. Most likely they should only have one regular generic user account (e.g. "cloud-user" on the default images provided by CSC) and the root user account.

SSH host keys (can be done with virt-sysprep)

Images used in the cloud must not contain any SSH host keys, as having them in the image would mean that every server launched using the image would have the same identity from the point of view of SSH. It is also a security risk, as anyone with access to the image file would be able to personate any server launched using that image file. Fresh SSH host keys need to be generated by cloud-init (see above) when a virtual machine first boots up.

Network interface ordering (can be done with virt-sysprep)

The udev device manager in the Linux kernel has a function that it pins a specific network interface name to a specific MAC address. This is not good if several virtual machines are to be created from an image, as all the virtual machines will have different MAC addresses. It is also not good if you create a snapshot out of a virtual machine and try to use that snapshot to launch a virtual machine, as it will remember the MAC address of the old virtual machine that was used to create the snapshot. The best way to do this is to use virt-sysprep.

Partitioning

When partitioning a Linux image, you should make sure the root partition is the first and only partition. During the virtual machine bootup process, OpenStack will insert SSH keys on the first partition under the /root/.ssh directory, which means this partition must be the root partition and not e.g. /boot. Logging in will not be possible without the root password unless the keys are correctly inserted.

ACPI daemon

The ACPI daemon is used to receive commands to manage the power state of a virtual machine. You should install an ACPI daemon on the machine images to allow proper power down/reboot from the cloud interface.

Hotplug

To be able to use volumes, you need to have ACPI hotplug enabled. This is on by default in CentOS 6 and newer, but for Ubuntu you need to add a line "acpiphp" to /etc/modules. For other distros, please check how to load acpiphp on boot from the distro documentation.

5.2 Uploading images

You can upload images either using the web interface or by using the "openstack" command line tool.

Before uploading, you will need to know what format the image you are uploading is. The most likely options are "qcow2" and "raw". You can find out the type of the image using the "file" command. This is what a qcow2 image looks like when you examine it with file:

$ file images/Ubuntu-15.10-Phoronix.qcow2
images/Ubuntu-15.10-Phoronix.qcow2: Qemu Image, Format: Qcow (v3), 10737418240 bytes

And this is what a raw image looks like:

$ file images/Ubuntu-14.04-old.raw
images/Ubuntu-14.04-old.raw: x86 boot sector; partition 1: ID=0x83, active, starthead 0, startsector 16065, 20948760 sectors, code offset 0x63

To upload using the command line, you would use this command:

openstack image create --disk-format <disk format> --private --file <image file to upload> <name of image to create>

This should upload the image. It will take a while before the image is usable.

If you prefer to use the web interface instead, you can upload images from the Compute | Images section by clicking the Create Image button:

You will be presented with this dialog:

In this example, we are creating an image called "Ubuntu". We have selected to provide a URL to download the image from by selecting "Image Location" for "Image Source". It is also possible to select a file from the local disk of the machine you are using the web interface with. We know that the image we are about to create is a qcow2 image, so we select that from the format dropdown menu. The other options don't need to be filled in. Also note that it is not possible for normal users to create public images. This will give an error message if attempted.

Previous chapter   One level up   Next chapter