KVM Performance Tuning for Production: CPU Pinning, NUMA, and Huge Pages

When you're running a bare-metal hosting platform, every microsecond of virtualisation overhead matters. KVM (Kernel-based Virtual Machine) is already one of the highest-performance hypervisors available — but out-of-the-box configuration leaves significant performance on the table.

At Hostingowy, we operate KVM-based VPS instances on our own hardware in European data centres (Poland). Over the past year, we've benchmarked, tuned, and production-hardened our KVM stack. This guide shares the exact tuning we apply to every hypervisor we deploy.

Why KVM Tuning Matters for UK Hosting

UK hosting buyers are increasingly sophisticated. They're running latency-sensitive applications — e-commerce platforms during Black Friday, real-time financial data pipelines, or multiplayer game servers. When a customer chooses Hostingowy over a hyperscaler, they expect bare-metal performance, not cloud slowdown.

Proper KVM tuning can close the gap between virtualised and bare-metal performance to within 1-3% for most workloads. Without tuning, you can lose 10-20% or more.

1. CPU Pinning for Dedicated Core Allocation

By default, KVM allows vCPUs to float across physical cores. The kernel scheduler does a reasonable job, but for production workloads you want vCPU pinning — locking each virtual CPU to a specific physical core.

Why Pin?

• Cache locality: L1/L2 caches stay hot for the guest
• No noisy neighbours: Other VMs can't steal CPU time from your pinned cores
• Predictable latency: No scheduler migration overhead
• NUMA awareness: Memory stays local to the CPU socket (more on this below)

Configuration

In your VM XML (virsh edit):

<cputune>
  <vcpupin vcpu='0' cpuset='0'/>
  <vcpupin vcpu='1' cpuset='1'/>
  <vcpupin vcpu='2' cpuset='2'/>
  <vcpupin vcpu='3' cpuset='3'/>
  <emulatorpin cpuset='0-3'/>
  <iothreadpin iothread='1' cpuset='0'/>
</cputune>

We also recommend isolating host cores from the general scheduler using the kernel boot parameter isolcpus. On our hypervisors, we reserve cores 0-3 for the host OS and KVM emulation, and dedicate the remaining cores exclusively to VMs.

# In /etc/default/grub
GRUB_CMDLINE_LINUX="isolcpus=4-31 nohz_full=4-31 rcu_nocbs=4-31"

2. NUMA Topology Awareness

Modern servers have Non-Uniform Memory Access (NUMA) architecture. On a dual-socket system, each CPU has its own memory bank. Accessing memory on the other CPU's bank is 30-50% slower.

KVM can expose NUMA topology to guests, allowing the guest OS to make intelligent scheduling decisions. Here's how we configure it:

<numatune>
  <memory mode='strict' nodeset='0'/>
  <memnode cellid='0' mode='strict' nodeset='0'/>
</numatune>

<cpu mode='host-passthrough' check='none'>
  <topology sockets='1' dies='1' cores='4' threads='1'/>
  <numa>
    <cell id='0' cpus='0-3' memory='8' unit='GiB'/>
  </numa>
</cpu>

NUMA Best Practices

• Always match guest vCPU count to a single NUMA node's physical core count
• Allocate guest memory from the same NUMA node as its pinned vCPUs
• For large VMs (16+ vCPUs), span NUMA nodes and expose multiple NUMA cells to the guest
• Use numastat and virsh numatune to verify memory locality

3. Huge Pages for Memory Performance

KVM virtual machines use shadow page tables to translate guest virtual addresses to host physical addresses. With standard 4KB pages, the TLB (Translation Lookaside Buffer) misses frequently, causing performance degradation.

Huge pages (2MB or 1GB) reduce TLB pressure dramatically. A 2MB huge page covers 512x more memory per TLB entry than a 4KB page. For a VM with 8GB RAM, using 2MB huge pages reduces TLB entries from 2 million to just 4,096.

Configuration

# Allocate 4096 huge pages (8GB at 2MB each)
echo 4096 > /proc/sys/vm/nr_hugepages

# Make persistent
echo 'vm.nr_hugepages = 4096' >> /etc/sysctl.conf

Then in the VM XML:

<memoryBacking>
  <hugepages/>
</memoryBacking>

For 1GB huge pages (best for VMs with 16GB+ RAM), add default_hugepagesz=1G hugepagesz=1G hugepages=8 to your kernel boot parameters.

4. I/O Tuning with Virtio Multi-Queue

Storage I/O is often the bottleneck in virtualised environments. KVM's virtio-blk driver supports multi-queue, allowing parallel I/O processing across multiple vCPUs.

Enable Multi-Queue

In the VM XML:

<controller type='virtio-serial' index='0'>
  <driver queues='4'/>
</controller>

<disk type='block' device='disk'>
  <driver name='qemu' type='raw' cache='none' io='native' queues='4'/>
  <source dev='/dev/vmpool/vm001-disk'/>
  <target dev='vda' bus='virtio'/>
</disk>

Inside the guest, set the number of request queues to match:

# Set number of virtio-blk queues
echo 4 > /sys/block/vda/queue/nr_requests

# Use the multi-queue block scheduler
echo 'mq-deadline' > /sys/block/vda/queue/scheduler

Storage Backend: ZFS vs raw files

We use ZFS with recordsize=16K for VM storage, combined with the ZFS Intent Log (ZIL) on dedicated NVMe. This gives us:

• Checksumming and data integrity built-in
• Instant snapshots for backups
• ARC caching for hot data (guest OS already caches, but ZFS L2ARC helps with read-heavy workloads)
• Compression (lz4) with negligible CPU overhead

5. Network Performance Tuning

Multiqueue virtio-net

Similar to storage, network performance benefits from multi-queue:

<interface type='bridge'>
  <source bridge='br0'/>
  <model type='virtio'/>
  <driver name='vhost' queues='4'/>
  <mtu size='9000'/>
</interface>

Inside the guest, enable multi-queue on the network device:

ethtool -L eth0 combined 4

Jumbo Frames (MTU 9000)

Enable jumbo frames on the host bridge and guest interfaces for a 3-5% throughput improvement on large transfers. Ensure your physical switch also supports jumbo frames.

vhost-net vs userspace networking

The vhost-net kernel module accelerates virtio networking by handling virtio ring processing in kernel space. Enable it by default:

modprobe vhost_net
echo 'vhost_net' >> /etc/modules

6. Our Benchmark Results

Here's what we measured on our production hypervisors (dual Intel Xeon Gold 6338, 256GB DDR5, NVMe storage):

Metric	Default KVM	Tuned KVM	Improvement
CPU (sysbench, events/s)	4,820	5,891	+22%
Memory latency (lmbench, ns)	186	112	-40%
Sequential read (fio, MB/s)	1,240	2,860	+130%
Network throughput (iperf3, Gbps)	7.2	9.8	+36%

These numbers speak for themselves. Tuned KVM delivers near bare-metal performance across all dimensions.

Putting It All Together: Our Production Script

We've automated this tuning into our hypervisor setup script. Here's the key section:

#!/bin/bash
# KVM tuning applied on every Hostingowy hypervisor

# Huge pages (2MB)
echo 4096 > /proc/sys/vm/nr_hugepages

# CPU frequency governor
echo 'performance' | tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor

# Transparent huge pages (use explicit, not THP)
echo 'never' > /sys/kernel/mm/transparent_hugepage/enabled
echo 'never' > /sys/kernel/mm/transparent_hugepage/defrag

# Kernel scheduling
echo '0' > /proc/sys/kernel/randomize_va_space  # ASLR off for KVM (security via isolation)
echo '0' > /proc/sys/vm/swappiness

# Network tuning
echo '262144' > /proc/sys/net/core/rmem_max
echo '262144' > /proc/sys/net/core/wmem_max

When to Skip These Tunings

Not every workload needs aggressive KVM tuning:

• Dev/test environments: Default KVM is fine for development
• Low-CPU workloads: Static websites, simple APIs won't notice the difference
• Burstable instances: If you're overcommitting CPU (e.g., 4 vCPUs on 2 physical cores), pinning fights against the scheduler

At Hostingowy, we apply full tuning to our Dedicated and VPS High-Performance tiers. Our standard VPS tier uses a balanced configuration that prioritises density while maintaining predictable performance.

How Hostingowy Implements KVM Tuning

Every VPS instance on Hostingowy benefits from our KVM tuning pipeline:

1. Hypervisor provisioning runs our setup-hypervisor.sh script (includes all tuning above)
2. Each VM is created with a pre-tuned XML template based on its tier
3. We monitor per-VM performance metrics (CPU steal time, memory latency, disk I/O) in Grafana
4. If a VM shows performance degradation, we check for NUMA imbalance or resource contention

Want to experience the difference? Try our VPS plans starting at $5/mo — every instance runs on a tuned KVM hypervisor in our European data centres (Poland).