Nested virtualization is a powerful feature that allows running a virtual machine inside another VM. This capability is crucial for testing, development, and lab environments where multiple layers of virtualization are required. Proxmox VE, a leading open-source virtualization platform, supports nested virtualization, enabling users to run hypervisors inside a Proxmox-based VM.
What is Nested Virtualization?
Nested virtualization is a technology that enables running virtual machines within virtual machines. Simply put, it’s like placing one box inside another. The physical host runs a virtualization software (e.g., VMware or KVM), which creates first-layer virtual machines (e.g., VMs on your computer). Nested virtualization allows installing another virtualization software within these first-layer VMs to create second-layer VMs, and even more layers. This technology breaks the traditional single-layer virtualization limitation, offering unprecedented flexibility.
To implement nested virtualization, both hardware and software must meet requirements. Hardware-wise, the CPU must support virtualization extensions. Software-wise, the host’s virtualization platform needs to enable nested virtualization support, and the first-layer VM configuration must be adjusted. However, nested virtualization incurs significant performance overhead. Each additional virtualization layer may degrade speed by 10% to 30%. For instance, second-layer VMs (L2) may perform noticeably slower than first-layer VMs (L1), particularly for compute-intensive tasks.
PVE as a Nested Hypervisor
PVE can:
Host Nested (Guest) Hypervisors: By default, PVE does not expose hardware-assisted virtualization extensions (such as Intel VT-x/AMD-V) to its virtual machines. As a result, guest hypervisors (Nested Hypervisors) cannot provide optimal performance for their internal VMs without additional configuration. To enable efficient nested virtualization, the VM's CPU type should be set to "host," and the physical PVE host must have nested virtualization extensions enabled.
Run as a Nested (Guest) Hypervisor: If PVE is running on another hypervisor and that hypervisor exposes hardware-assisted virtualization extensions to PVE, then PVE can utilize these extensions to improve the performance of its internal VMs. Otherwise, in cases like PVE-inside-PVE (where PVE runs inside another PVE instance), the VMs inside the nested PVE can only run if KVM hardware virtualization is disabled.
Note: VMs with nested virtualization enabled (vmx/svm flags) cannot be live migrated!
How to Enable Nested Virtualization in PVE?
Check if PVE Virtual System Supports Virtualization
By default, the CPU of a VM created in PVE does not support VMX, meaning nested virtualization is not enabled. You can verify this inside the VM using the following command:
egrep --color 'vmx|svm' /proc/cpuinfo
If there is no output, nested virtualization is not supported. If “vmx” or “svm” is highlighted, it is supported.
Steps to Enable Nested Virtualization
1. Enable nested virtualization on the PVE host and shut down all VMs
Check if nested virtualization is enabled on the PVE system by running:
cat /sys/module/kvm_intel/parameters/nested
If the output is “Y”, it is already enabled.
If the output is “N”, it is not enabled.
If nested virtualization is not enabled, you must shut down all virtual machines before proceeding, as enabling kernel support requires all VMs to be turned off.
Run the following commands to enable nested virtualization:
modprobe -r kvm_intel
modprobe kvm_intel nested=1
Verify if it has been successfully enabled:
cat /sys/module/kvm_intel/parameters/nested
If the output is “Y”, nested virtualization is enabled.
If you encounter the error “Module kvm_intel is in use”, ensure all VMs are shut down before retrying.
2. Automatically enable nested virtualization on system boot
To make nested virtualization persist after a system reboot, add the following line to the configuration file:
echo "options kvm_intel nested=1" >> /etc/modprobe.d/modprobe.conf
This ensures that nested virtualization is automatically enabled after a restart.
3. Set the VM CPU type to "host"
qm set <vmid> --cpu cputype=host
Alternatively, you can configure this through the PVE GUI:
Select the VM - Hardware - Processor - Type - host
Note: Nested virtualization is enabled by default in PVE7
Backup Your Proxmox VMs with the Safest Way
Vinchin Backup & Recovery is a powerful data protection solution designed for virtualized environments, including Proxmox host-based virtual machines. When backing up Proxmox virtual machines with Vinchin, users benefit from efficient backup and recovery features, utilizing incremental backups, deduplication, and compression technologies to significantly reduce storage usage while ensuring fast and stable backup processes. Its intuitive interface and automated backup tasks allow Proxmox users to easily manage large-scale VM backups and quickly restore them in case of failure, ensuring business continuity and data security.
It only takes 4 steps for you to backup Proxmox VE VMs:
1.Select the backup object.
2.Select backup destination.
3.Configure backup strategies.
4.Review and submit the job.
It offers a wide range of features, including V2V migration, allowing seamless transfers between different virtual platforms. Click the button below to download a 60-day free trial and experience it for yourself!
Proxmox Nested Virtualization FAQs
1. Can I Run Proxmox Inside a Proxmox VM?
Yes, once nested virtualization is enabled, you can install Proxmox as a guest VM on a Proxmox host.
2. Are There Any Security Concerns with Nested Virtualization?
Running nested virtualization can introduce security concerns, especially if the nested VMs are not properly isolated. Vulnerabilities in the virtualization layer can propagate through multiple layers. It's important to apply security patches regularly and maintain proper isolation between VMs.
Conclusion
Nested virtualization in Proxmox VE enables powerful multi-layered virtual environments, making it ideal for testing and development. While it requires proper hardware support and configuration, it significantly enhances flexibility. However, users should be aware of performance trade-offs and the inability to live migrate VMs with nested virtualization enabled. By following the outlined steps, you can successfully enable and optimize nested virtualization in PVE.
