Many companies pay for virtualization twice: once in commercial hypervisor licenses, and again in dependence on a single vendor who can change pricing, the subscription model, or end product support at any time. Meanwhile, every Linux server already contains a ready, production-proven hypervisor – KVM – which powers a large share of public clouds, yet it is often installed "quickly" following guides written several system versions ago. The result is predictable: commands copied from the internet fail (because a package has disappeared from the repositories), the host runs an outdated monolithic daemon instead of the new architecture, virtual machines "vanish" because the user connects to a different context than root, and a careless network bridge configuration can cut off remote access to the server. The solution is to treat virtualization like any other infrastructure component: understand the layers (kernel, emulator, management layer, interface), deliberately configure permissions and networking, and only then create virtual machines. This model – KVM as a kernel module, QEMU as the VM process, libvirt as the management API – is the same on RHEL, AlmaLinux, Rocky, Fedora, Debian and Ubuntu. In this article, I first explain how it works in a distribution-independent way, and then walk through a complete step-by-step deployment on AlmaLinux 10 / RHEL 10 – from package installation, through modular libvirt daemons and permissions, to the Cockpit web console, testing and an emergency plan.
Why should you read this article?
Zero hypervisor licensing costs. KVM is part of the Linux kernel, and libvirt, QEMU and Cockpit are available in the standard repositories – you pay only for hardware and (optionally) an OS support subscription.
No vendor lock-in. Virtual machines stored as qcow2 images and libvirt XML definitions can be moved between any Linux distributions, and even to KVM-based platforms (Proxmox, OpenStack, OpenShift Virtualization).
An up-to-date procedure for RHEL 10, not RHEL 7. I show which commands from popular guides no longer work (including
bridge-utils, the monolithiclibvirtdand SPICE) and what to replace them with, before your installation fails halfway through.Protecting access to the host. Permission settings and network bridge configuration are the two places where a single typo cuts off sudo or SSH – the procedure includes backups, testing without closing your session, and an automatic rollback of changes.
Portable architectural knowledge. Understanding the "client → URI → socket → polkit → daemon → QEMU process" chain lets you diagnose problems on any distribution instead of just replaying commands.
What you will learn
how KVM, QEMU, libvirt and Cockpit work together and what each layer is responsible for,
how the modular libvirt daemons (
virtqemud,virtnetworkd…) differ from the monolithiclibvirtd, and why you should choose the former on RHEL 10,why AlmaLinux 10 / RHEL 10 was chosen as the reference environment and which commands from older guides need to be corrected,
how to configure user permissions, the NAT network, an optional
br0bridge and the Cockpit console without risking loss of access,how to safely test the hypervisor, roll back a failed change and diagnose the most common errors (SELinux, polkit, networking,
qemu:///session).
Table of contents
Universal part: how KVM, QEMU and libvirt work
"KVM" in everyday usage is actually a stack of four layers. Each has a different responsibility and a different set of files, and most problems come from confusing which layer the error lives in.
Layer | Component | Responsibility |
|---|---|---|
Kernel |
| Using hardware virtualization extensions (Intel VT-x / AMD-V), isolating guest memory and CPU |
Emulator | QEMU ( | One process per virtual machine, device emulation (disks, network cards, VNC console) |
Management | libvirt ( | Stable API, VM definitions in XML, networks, storage pools, permissions, autostart |
Interface |
| Tools that talk to libvirt – never directly to QEMU |
Operating model step by step
The administrator issues a command (e.g.
virt-install,virsh start, a click in Cockpit).The client tool determines the connection URI – most often
qemu:///system(system VMs managed by a daemon running as root) orqemu:///session(private VMs of a given user).The client connects to the libvirt daemon's UNIX socket (e.g.
/run/libvirt/virtqemud-sock). Thanks to systemd socket activation, the daemon does not need to run constantly – it starts on the first connection.The daemon checks authorization – in the RHEL family, by default via polkit. A member of the
libvirtgroup gets full management rights without entering a password.virtqemudgenerates the configuration and starts the QEMU process as theqemuuser, with an SELinux (sVirt) label unique to that VM.QEMU opens
/dev/kvmand hands guest code execution over to the kernel, which uses the CPU's hardware extensions.Helper daemons handle the rest:
virtnetworkdcreates thevirbr0bridge with NAT and DHCP (dnsmasq),virtstoragedmanages storage pools,virtnodedevdmanages host devices.
Common system layer: files, services and sockets
Element | Path / unit | Role |
|---|---|---|
KVM device |
| Kernel interface used by QEMU |
VM definitions (system) |
| Persistent VM definitions – edit only via |
Network definitions |
| Virtual networks (e.g. |
Daemon configuration |
| Modular daemon settings |
Client configuration |
| Including the default URI ( |
Disk images |
| The |
VM logs |
| QEMU process output – the first place to look when a VM fails to start |
Daemon sockets |
| On-demand daemon activation |
polkit rule |
| Permissions for the |
Web console |
| Cockpit interface with the |
Key dependencies and architectural pitfalls
qemu:///systemvsqemu:///session. Without extra configuration, a regular user connects toqemu:///session. VMs created in a user session are not visible to root or Cockpit (and vice versa), have no access to the NATdefaultnetwork and do not start with the host. This is the most common source of the question "where did my VM go?".The
libvirtgroup is root-equivalent. Anyone who can define a VM can attach any host disk or directory to it. Grant membership in this group as restrictively aswheel.SELinux and image location. Images outside
/var/lib/libvirt/images(e.g./data/vms,/home) require thevirt_image_tcontext – otherwise the VM fails to start with "Permission denied", even with correct Unix permissions.Host network vs bridge. The
defaultnetwork (NAT,virbr0) does not touch the physical interface and is safe. Abr0bridge "takes over" the physical NIC – a mistake in its configuration on a remote server means losing SSH.Hardware support. Without VT-x / AMD-V enabled in BIOS/UEFI, QEMU runs in emulation mode, many times slower. In a nested environment (the host is itself a VM), nested virtualization must be enabled.
Tip: Before you start diagnosing "KVM doesn't work", ask yourself three questions in this order: (1) does
/dev/kvmexist, (2) which URI is my client connecting to (virsh uri), (3) what does the log/var/log/libvirt/qemu/<name>.logsay. These three steps solve most problems.
Differences between distributions
Family / system | Packages / tools | Shared files / mechanisms | What to watch out for |
|---|---|---|---|
RHEL 10 / AlmaLinux 10 / Rocky 10 |
| Modular daemons by default, SELinux (sVirt), polkit, NetworkManager | No |
RHEL 9 / AlmaLinux 9 | As above | Modular daemons by default on fresh installs | After upgrading from version 8, the host may still use |
Fedora | As above, newer QEMU/libvirt versions | Modular daemons, SELinux | Fast release cycle – good for testing, not for production |
Debian / Ubuntu |
| AppArmor instead of SELinux, | The daemon type depends on the release – check |
openSUSE / SLES |
| Modular daemons by default | Different default MAC protections and network configuration paths |
Deployment example: preparing AlmaLinux 10 and RHEL 10
Why AlmaLinux 10
Reason | What it means in practice |
|---|---|
Compatibility with RHEL 10 | The same packages, paths and procedures as in Red Hat subscription environments – you apply RHEL documentation 1:1 |
Long support cycle | The 10 branch is supported until 2035 – the hypervisor does not need to be migrated every few years |
No licensing fees | You can build a lab and non-production environments without subscription costs, and move to RHEL with support if needed |
Up to date compared with guides | Most online guides describe RHEL 7/8 ( |
Variant for older hardware | In addition to the default x86-64-v3 architecture, AlmaLinux provides an x86-64-v2 variant, so older servers can be used as test hosts |
Prerequisites
Step 1. Check the system version:
cat /etc/os-release | grep -E Step 2. Confirm the CPU has virtualization extensions (result greater than 0):
grep -cE Step 3. Check resources – the minimum is 2 cores, 4 GB RAM and a few GB of free space per VM:
nproc
free -h
df -h /var/lib
Step 4. Make sure repositories are available and the system is up to date:
sudo dnf repolist
sudo dnf upgrade -y
Step 5. Check your user and sudo rights (save the output of id – you will need it for comparison):
id
sudo -v && Step 6. Confirm the SELinux state (expected: Enforcing) and the firewall:
getenforce
sudo firewall-cmd --state
Installing packages
Package | Role | Required? |
|---|---|---|
| QEMU emulator with KVM support | Yes |
| Management API and modular daemons | Yes |
| Full set of libvirt drivers for KVM | Recommended |
| Definition of the NAT | Recommended |
| Creating VMs from the command line | Yes |
| Graphical console client (VNC) | Optional |
| Web console with the virtual machines module | Recommended |
| Declarative network configuration with automatic rollback ( | Optional |
| The old | No – not available in RHEL 8+ repositories |
# Install the full virtualization stack on AlmaLinux 10 / RHEL 10Equivalent for Debian / Ubuntu:
sudo apt update
sudo apt install -y qemu-system-x86 libvirt-daemon-system libvirt-clients \
virtinst virt-viewer cockpit cockpit-machines
Note: Many guides add the
bridge-utilspackage to the list. On RHEL 10 and AlmaLinux 10 it is not in the base repositories, and when a package is missingdnfaborts the entire transaction – nothing gets installed. Handle bridges with theip/bridgecommands (iproutepackage),nmcliornmstatectl. Also keep in mind that RHEL 10 no longer supports the SPICE protocol (the graphical console uses VNC), thei440fxmachine type is deprecated (useq35), and some old CPU models have been removed.
Verifying the installation
# KVM kernel modules are loadedAll key virt-host-validate items should show PASS. A WARN status for IOMMU matters only if you plan to pass PCI devices through to VMs.
Host preparation: daemons, permissions and network
Pre-launch conditions
virt-host-validate qemureports noFAILfor hardware virtualization.You have a second SSH session with root privileges open (or console access: IPMI, iDRAC, iLO, the parent hypervisor's console).
You know the name of the physical network interface and its current configuration (
nmcli device status).You have saved the output of
id– after changing groups you will compare it with the new state.You know whether the host was upgraded from RHEL 8/9 (in that case it may still use the monolithic
libvirtd).
Starting libvirt daemons and granting permissions
Primary step – modular daemons (recommended on RHEL 10). Each libvirt area has its own daemon and three sockets (-sock, -ro, -admin). Enable them persistently:
# Enable and start the sockets of all modular libvirt daemonsFallback option – monolithic libvirtd. The old approach from popular guides (systemctl enable --now libvirtd) still works on RHEL 10, but libvirtd is deprecated and will be removed in the next major release. The modular units and libvirtd are mutually exclusive – never run both at once.
# ONLY if you deliberately choose monolithic mode (e.g. compatibility with old software)Option | Behavior | When to choose |
|---|---|---|
Modular daemons ( | A separate process per area, on-demand activation, restarting one daemon does not affect the others | Every new RHEL 9/10 installation and derivatives |
Monolithic | A single process handles all drivers | Only temporarily – a host upgraded from RHEL 8 or tools that still require it |
User permissions. Add the administrator to the libvirt group (managing system VMs) and kvm (access to /dev/kvm for user-session VMs):
# -a (append) is CRITICAL – without it you remove the user from all other groups, including wheelWarning: The command
usermod -G libvirt "$USER"(without-a) replaces the group list instead of extending it. The user loses membership inwheel, and with it sudo. Before making the change, keep a root session open and back up the group files:sudo cp -a /etc/group /root/group.$(date +%F-%H%M)andsudo cp -a /etc/gshadow /root/gshadow.$(date +%F-%H%M).
New group membership only takes effect in a new login session – log out and log back in (just opening a new terminal tab in an existing graphical session is not enough).
Verifying configuration artifacts
# In a NEW session: were the groups added, and is wheel still thereCore configuration: libvirt, network and Cockpit
Guiding principle: backup and a way back first, then the change, and finally a test in a new session. This applies especially to networking – NAT configuration is safe, a bridge on the physical interface is not.
Safety rules before editing
Timestamped backup before every change – the
/etc/libvirtdirectory, network definitions and NetworkManager profiles.An open emergency session – a second SSH session as root, or out-of-band console access (IPMI/iDRAC/iLO), which you keep open until testing is complete.
Narrow the scope – grant
libvirtpermissions to specific, named administrator accounts, never to service accounts or entire domain groups "just in case".No manual editing of files managed by libvirt – change the XML files in
/etc/libvirt/qemu/viavirsh editandvirsh net-edit; manual edits will be overwritten or ignored.Network changes with automatic rollback – deploy a bridge on the physical interface with
nmstatectl apply --no-commit, so a mistake reverts itself.
# Back up libvirt and network configuration before making changesConfiguring service A: libvirt and the VM network
1. Default URI for the administrator. To make virsh run without sudo manage the same VMs as root and Cockpit, set the default system connection:
mkdir -p ~/.config/libvirt
2. The NAT default network. Gives VMs internet access through the host without touching the physical interface:
virsh net-start default 2>/dev/null || Full reference definition of the default network (view it with virsh net-dumpxml default):
<Warning: Do not blindly copy this definition onto a host whose LAN already uses the
192.168.122.0/24range – a routing conflict will cut off either the VMs or the host itself. In that case, change the addressing withvirsh net-edit default, then runvirsh net-destroy default && virsh net-start default.
3. Optional: a br0 bridge with rollback. Needed when VMs should get addresses from the physical network (like regular servers on the LAN). Instead of brctl, we use nmstate, which automatically restores the previous state on failure. First check the interface name (nmcli device status), then prepare the file:
# Replace enp1s0 with the name of your physical interfaceIn a new SSH session (to the new address, if DHCP assigns a different one), check connectivity and only then commit the change:
ip -br addr show br0
bridge link show
ping -c 3 If the new session does not connect, do nothing – once the timeout expires, the configuration reverts to its previous state. To set a static address, replace dhcp: true in the same file with an address: section containing the address and prefix.
Configuring service B: Cockpit (web console)
Cockpit provides graphical VM management (the Virtual Machines tab) and, on RHEL 10, is the official successor to the deprecated virt-manager. By default it listens on port 9090/tcp over HTTPS.
# Back up the Cockpit configuration (the directory may contain only disallowed-users)Local option (safest). Do not open the port in the firewall; instead connect through an SSH tunnel from your workstation and open https://localhost:9090/:
# On the administrator's workstationNetwork option – administrative subnet only. Instead of opening Cockpit to the whole network, restrict access with a rich rule:
# Remove general access (if it was enabled in the zone profile)Login: use your own account from the wheel group (and libvirt), and switch the console to administrative access mode. The root account is blocked by default in /etc/cockpit/disallowed-users – and it should stay that way.
Example reference file /etc/cockpit/cockpit.conf (create it only if you need to change the default behavior):
[WebService]Warning: Do not blindly add options found in guides, such as
AllowUnencrypted = trueorAllowMultiHost = yes. The first disables session encryption; the second enables the host switcher, which is deprecated in RHEL 10 and disabled by default precisely for security reasons.
Service | Position in the chain | Effective order of steps |
|---|---|---|
| Client | URI from |
| libvirt network layer |
|
| Host network layer | NetworkManager (nmstate) → |
Cockpit | Web interface | Browser → firewalld (rich rule or SSH tunnel) → |
Explaining the condition and rule logic
How the client chooses a URI – the first matching rule wins:
An explicit
--connect/-cparameter (e.g.virsh -c qemu:///system list).The
LIBVIRT_DEFAULT_URIenvironment variable.The
uri_defaultentry in~/.config/libvirt/libvirt.conf(for root:/etc/libvirt/libvirt.conf).The default value:
qemu:///systemfor root,qemu:///sessionfor a regular user.
A typo in step 3 (e.g. qemu:///sytem) causes a connection error on every virsh command – easy to spot. A missing file is more dangerous: commands work, but in the user session, so the administrator "can't see" production VMs or creates them in the wrong context.
How the daemon decides on access:
The
virtqemud-socksocket accepts the connection and passes on the client's identity (process UID, GID).polkit evaluates the
org.libvirt.unix.manageaction – the rule shipped with libvirt returns "yes" for members of thelibvirtgroup.A user outside the group gets an administrator authentication prompt or a denial (
authentication unavailable).The
-rosocket provides read-only access – useful for monitoring that should not be able to change anything.
Risk of typos: omitting -a in usermod removes the user from wheel; a wrong interface name in br0.yml creates a bridge without a physical port (the host loses networking if you move the address to br0); a mistake in the subnet address of the rich rule will block Cockpit for administrators or open it to the wrong network. Each of these mistakes is reversible, provided you have a backup and an emergency session.
Testing, emergency plan and common problems
Testing without cutting off access
Run this in a new SSH session, without closing the root emergency session:
# 1. Groups and sudo in the new sessionFinally, log in to Cockpit (through the tunnel or from the administrative subnet) and check that the Virtual Machines tab shows the default network and the image pool. Close the emergency session only after all steps pass.
Restoring access: emergency plan
Step 1. Loss of sudo after usermod without -a – in the open root session (or via the IPMI/iDRAC console as root), restore the groups:
usermod -aG wheel,libvirt,kvm admin
Step 2. Loss of networking after bridge configuration – if nmstatectl commit was not run, wait for the timeout (rollback is automatic). If the change was committed, log in via the out-of-band console:
nmcli connection show
nmcli connection delete br0
tar xzf /root/nm-connections-YYYY-MM-DD-HHMM.tar.gz -C /
nmcli connection reload
nmcli connection up Step 3. Broken libvirt configuration – restore the directory and restart the daemons (running VMs will not be stopped – QEMU processes are independent of the daemon):
tar xzf /root/libvirt-etc-YYYY-MM-DD-HHMM.tar.gz -C /
systemctl restart virtqemud.service virtnetworkd.service
Step 4. Broken default network – recreate it from the XML backup:
virsh net-destroy default
virsh net-undefine default
virsh net-define /root/net-default-YYYY-MM-DD-HHMM.xml
virsh net-start default
virsh net-autostart default
Step 5. No access to Cockpit – the console is not the only path: manage VMs over SSH with virsh, and revert the firewall rule with firewall-cmd --permanent --remove-rich-rule='...' and firewall-cmd --reload.
Common problems (troubleshooting)
Symptom | Cause / solution |
|---|---|
| The package does not exist in RHEL 8+. Remove it from the list – bridges are handled by |
| The client connects to |
| The user is not in the |
| VT-x/AMD-V disabled in BIOS/UEFI, or the host is a VM without nested virtualization. Enable it in firmware or in the parent hypervisor |
VM does not start: "Permission denied" when accessing the image | SELinux – the image is outside |
"Network 'default' is not active" |
|
Importing an old VM: "spice graphics are not supported" | RHEL 10 does not support SPICE. |
An old VM does not start after migrating to RHEL 10 (machine type or CPU model) |
|
Conflict between | Both modes are running. |
Cockpit unreachable from the network |
|
No "Virtual Machines" tab in Cockpit | The |
Cockpit login as root rejected | Intended behavior ( |
General libvirt daemon error |
|
Quick deployment checklist
AlmaLinux 10 / RHEL 10 updated, SELinux in
Enforcingmode.CPU reports
vmxorsvm, virtualization enabled in BIOS/UEFI.Packages installed:
qemu-kvm,libvirt,virt-install,cockpit-machines(withoutbridge-utils).virt-host-validate qemuwith noFAILstatuses.Modular daemon sockets enabled,
libvirtdinactive.Backups of
/etc/group,/etc/gshadow,/etc/libvirt, NetworkManager profiles and thedefaultnetwork made.Administrator added to
libvirtandkvmviausermod -aG,wheelretained.uri_default = "qemu:///system"set,virsh urireturnsqemu:///system.defaultnetwork active with autostart, no address conflict.(Optional)
br0bridge deployed vianmstatectl apply --no-commitand committed after testing.Cockpit reachable via SSH tunnel or only from the administrative subnet, root login blocked.
smoke-testVM started withtype='kvm'and removed.Emergency session closed only after all tests pass.
FAQ
Can I still use systemctl enable --now libvirtd, as in older guides?
Yes, on RHEL 10 it still works, but libvirtd is deprecated and will be removed in the next major release. On new installations use the modular daemons – it doesn't change how you work with virsh, Cockpit or VM definitions, and it removes a future migration.
Does this procedure work on Debian and Ubuntu?
Yes, the architecture is identical – what changes are the package names (qemu-system-x86, libvirt-daemon-system, virtinst), the MAC mechanism (AppArmor instead of SELinux) and how networking is configured. The libvirt and kvm groups, URIs, virsh, virt-install and Cockpit work the same way.
Is membership in the libvirt group safe?
No safer than sudo. A user in this group can attach any host disk or directory to a VM, so in practice they have root privileges. Grant it only to named administrator accounts, and use the read-only socket (virtqemud-sock-ro) for monitoring.
Does virt-manager still work on RHEL 10?
Yes, but it is deprecated, and Cockpit is its successor. If you prefer a desktop client, run virt-manager on your workstation and connect remotely to the host via qemu+ssh://admin@host/system – the host then needs no graphical environment.
How does this setup differ from Proxmox VE?
Proxmox is a ready-made distribution with KVM, its own API, clustering and backup in a single package. Plain KVM with libvirt on RHEL/AlmaLinux gives you fewer features out of the box, but full compatibility with the enterprise ecosystem (SELinux, subscriptions, Red Hat tooling, Ansible) and no extra vendor layer. You can move qcow2 images between both platforms.
Can I run KVM inside a virtual machine (nested virtualization)?
Yes, provided the parent hypervisor exposes VT-x/AMD-V extensions to the guest (e.g. the host-passthrough CPU model). It is a good solution for labs and procedure testing, but not for production – performance and vendor support are limited.
How do I scale this solution to many hosts?
Start with automation: the same procedure written as an Ansible role gives you repeatable hosts, and Cockpit and virsh support remote connections. With a dozen or more hosts and a need for live migration, high availability or self-service, consider a KVM-based management platform (e.g. OpenNebula, OpenStack or OpenShift Virtualization) – the VM format stays the same, so you don't lose your investment.