An Experimental Analysis of the Xen and KVM Latencies

This artifact is composed of Makefiles and scripts that can be used to measure the latency experienced in KVM-based and Xen-based VMs (latency measurements are performed by using cyclictest).

In particular, these scripts have been used to perform the experiments presented in the paper and can be used to reproduce similar experiments on different hardware/software configurations.

Description in PDF Format


The isorc-artifact.tgz tarball contains the scripts and Makefiles to be used to reproduce the experiments.


Although the artifact has been tested on various Intel and AMD CPUs (including an Intel Core Duo, various versions of Intel Core i7 and i5, several Intel Xeon, an AMD Ryzen Threadripper, and some other), every CPU supported by KVM and Xen should be usable.

The achieved results will be comparable with the ones presented in the paper (and consistent with them).

The Makefiles use the same computer (the host) to compile the needed software and to run the experiments. Hence, both the tools needed for compilation and the two hypervisors under testing have to be installed on the host. This requires to install a Linux distribution on the host (every recent Linux distribution should work; Debian stable, Debian testing, Ubuntu 18.04, Linux Mint and OpenSUSE have been tested).

The tools needed for compilation are GNU make, gcc, binutils, flex, bison, bc, and can be installed by using the package manager provided by the host distribution. Moreover, to compile the Linux kernel the "libelf" and "libssl" libraries (including the header files) must be installed. On a Debian, Mint, or Ubuntu system they can be installed with "apt-get install libelf-dev libssl-dev".

The hypervisors needed to run the experiments are QEMU/KVM and Xen and are assumed to be installed on the host distribution. Both of them can be installed by using the package manager provided by the host distribution. However, some distributions provide a version of Xen that is too old for these tests (Xen 4.11 has been used in the paper). In this case, Xen can be installed from source as indicated in the Xen wiki (note that running "make install" is not recommended; use "make debball" or "make rpmball" instead, and then use the package manager).

Since some of the experiments compare the latencies experienced when using an RT host kernel (Linux with the Preempt-RT patches) with the ones experienced when using an NRT kernel (vanilla Linux), a Preempt-RT kernel (4.19.8-rt6 has been used in the paper) and a vanilla Linux kernel (as provided by the host distribution) must be installed on the host. The artifact provides a way to build a basic Preempt-RT kernel for the host (see below).

Some scripts and Makefiles use the "sudo" command, that must also be installed in the host (if it is configured not to require a password, the execution of the experiments is simplified).

Finally, the "gnuplot" program is used to generate the figures and must be installed in the host (the package manager provided by the host distribution can be used).


The only software components that the artifact does not automatically install are the hypervisors (see the previous section) and the host's kernel. If you want to reproduce the experiments from the paper and a Preempt-RT kernel is not already installed on the host, it is possible to install a basic Preempt-RT kernel by following these instructions:

Notice that the configuration used to compile the Preempt-RT kernel is a simple one that should boot on standard PCs, if you use a non-Intel host or if you have particular devices (laptops often fall in this category) you might need to change the configuration in "BuildHost/.config" and recompile the kernel.

After this, the artifact is usable.

Running the Experiments

The latency experiments are driven by some Makefiles: the Makefiles in Artifact/LatencyTests allow to run generic latency measurements on the host or in KVM-based or Xen-based VMs, while Artifact/Makefile can be used for reproducing the runtime conditions of some experiments presented in the paper.

For example, by typing "make exp12" in the "Artifact" directory, it is possible to produce a figure similar to Figures 1 and 2 of the paper, running the same experiments. Basically, this command uses "Artifact/LatencyTests/VMTest" to measure the latencies in Xen and KVM machines with various host and guest kernels (RT and NRT). Before starting the experiment, the Makefile checks if the host configuration (kernel and hypervisor) is correct, and if the configuration is not correct the Makefile prints a message (asking to reboot the host) and stops. To run all the experiments, 3 or 4 reboots are needed (to get the appropriate host configurations: no Xen NRT, no Xen RT, Xen NRT and Xen RT). The experiments are performed in a pre-defined order (KVM NRT host / NRT guest, KVM NRT host / RT guest, KVM RT host / NRT guest, KVM RT host / RT guest, Xen NRT host / NRT guest, etc...).

By typing by typing "make exp3" or "make exp4" in the "Artifact" directory it is possible to produce figures similar to Figure3 and Figure 4 of the paper.

The Artifact/LatencyTests/VMTest Makefile is more powerful (allowing to run different kinds of experiments), but more complex to be used. The simplest way to invoke it is by typing "make cdf-xen-<id>.txt", for measuring the latencies in a Xen VM, or "make cdf-kvm-<id>.txt", for measuring the latencies in a KVM VM (here, <id>" is just an identifier for the experiment, at the user choice). These commands automatically compile a Preempt-RT kernel for the guest, build a simple guest image (a simple initramfs containing busybox, cyclictest, and some scripts for running the experiment), run the experiment, and compute the eCDF of the latencies experienced on the guest.

To run "make cdf-xen-<id>.txt", the host must have been booted with the Xen hypervisor, and to run "make cdf-kvm-<id>.txt" the host must have been booted without the Xen hypervisor; if these prerequisites are not respected, the test fails. The Makefiles in the Artifact directory take care of verifying these prerequisites and notifying the user (asking to reboot with or without Xen, or with the correct host kernel) instead of just failing.

Running "make xen-test" or "make kvm-test" the experiment will be repeated 10 times.

The experiments are performed using an RT guest kernel, to run them with an NRT kernel, you can run "make bzImage-nrt; cp bzImage-nrt bzImage" before starting the experiments. The host kernel, instead, is selectable at boot time.

Using "make cdf-xenstress-<id>.txt" instead of "cdf-xen-<id>.txt" (or "make cdf-kvmstress-<id>.txt" instead of "cdf-kvm-<id>.txt" it is possible to run the experiments with some background kernel stress load on the host (or on Dom0), used to trigger higher latencies.

Finally, Artifact/LatencyTests/Makefile allows to run some latency measurement on the host kernel (or on the Dom0 kernel, if the host has been booted with the Xen hypervisor), with or without background stress. This Makefile is used by the "exp3" and "exp4" targets of Artifact/Makefile.

Expected Results

After successfully running "make exp12" (or similar) in the "Artifact" directory, several files appear in the "Figure12" (or similar) directory. Notice that it might be necessary to run "make exp12" and reboot several times; after the last time, the "make" command is successful, and an eps file is generated. This file is consistent with the corresponding figure in the paper.

The generated text files (for example, "Figure12/cdf-*-*-*.txt" contain the results of the experiments executed for the various host/guest configurations.

As said, the obtained results are consistent with the ones presented in the paper, but the numbers cannot be equal. For example, looking at the results in "Figure12" it is possible to notice that:

Finally, notice that some details of the figures in the paper (for example, the X scale and the frequency of displayed points) have been adjusted to make the figures more readable. This manual tuning requires to see the values generated by the experiments and is not performed on the figures generated by the artifact's Makefile (for example, the maximum X value on the figures has been set to an arbitrary value --- 1000us for Figure12). The figures can be tuned by editing the various "" files contained in the "Figure*" directories.

Customization of the Experiments

The experiments can be easily customized by modifying the test scripts used in the guest (in the "Artifact/LatencyTests/VMTest/GuestScripts" directory) or the Makefiles. Modifying the guest scripts allow changing the cyclictest parameters (for example, the cyclictest period, or duration) while modifying the host Makefiles allow to change the VM setup (for example, creating a VM with multiple CPUs, or changing the VM scheduling).