OpenStack is an open-source cloud computing platform whose architecture includes compute, storage, and networking services, providing powerful virtualization capabilities and automated management functions. To ensure the high availability of the OpenStack platform, certain architectural solutions and technical measures need to be adopted. This article will introduce some common OpenStack high availability architecture solutions.
OpenStack High Availability
To understand how to achieve high availability, it is essential to identify which services are prone to unAvailability. First, let’s get a basic understanding of OpenStack’s structure.
OpenStack consists of five major components: Compute (Nova), Identity Management (Keystone), Image Management (Glance), Front-end Management (Dashboard), and Object Storage (Swift).
Nova is the core component responsible for computing and control. It includes services such as nova-compute, nova-scheduler, nova-volume, nova-network, and nova-api. Like most other distributed systems, OpenStack is divided into two types of nodes: control nodes and compute nodes. Control nodes provide all services except for nova-compute. These components and services can be installed independently, and combinations can be chosen as needed.
Nova-compute runs on each compute node. For now, let’s assume it is trustworthy; otherwise, a backup machine can be used for failover (though configuring a backup for each compute node may be too costly compared to the benefits).
High Availability of the Control Node
Ensuring the high Availability of the control node is the primary challenge, and different components have their own specific Availability requirements and solutions.
(1) What happens if the Control Node fails, considering there is only one Control Node managing and controlling the entire system?
This is the common single point of failure (SPoF) issue. High availability cannot be achieved with a single machine alone. More often, a solution is designed to ensure a rapid takeover of the failed machine when an issue arises, though this comes at a higher cost.
To address the single-point issue, the typical solution is to use redundant devices or hot standby. Since hardware failures or human errors can always cause one or multiple node failures, and maintenance or upgrades sometimes require temporarily stopping certain nodes, a reliable system must be able to withstand the stoppage of one or multiple nodes.
Common deployment modes include: Active-passive standby mode, Active-active mode and Cluster mode
(2) How to build redundant control nodes? Or what other methods can achieve highly reliable control?
Many may consider implementing an active-passive mode, using a heartbeat mechanism or similar methods for backup, and achieving high Availability through failover. OpenStack does not inherently support multiple control nodes, so Pacemaker requires various services to implement their own backup, monitoring, and switching mechanisms.
A closer look at the services provided by the control node shows that they mainly include nova-api, nova-network, nova-scheduler, nova-volume, as well as Glance, Keystone, and the database MySQL. These services are provided separately. Nova-api, nova-network, and Glance can run on each compute node, RabbitMQ can operate in a primary-backup mode, and MySQL can use a redundant high-availability cluster.
The following sections detail these components.
High Availability Solutions for Key Components
1) High Availability of nova-api and nova-scheduler
Each compute node can run its own nova-api and nova-scheduler, ensuring load balancing for proper operation. This way, when the control node fails, the nova-api and related services on compute nodes continue to function normally.
2) High Availability of nova-volume
Currently, there is no perfect HA solution for nova-volume, and more work is needed. However, since nova-volume is driven by iSCSI, integrating it with DRBD or using a high-availability hardware solution based on iSCSI can achieve high Availability.
3) High Availability of Networking Services (nova-network)
OpenStack already provides multiple high-availability networking solutions. A commonly used approach is to enable high availability mode by using the “Multi-host” option. Other common solutions include Failover, Multi-nic, Hardware Gateway, and Quantum.
4) High Availability of Glance and Keystone
OpenStack images can be stored using Swift, and Glance can run on multiple hosts. Pacemaker is a powerful high-availability solution for managing multi-node clusters, implementing service switching and failover, and can be used with Corosync and Heartbeat. Pacemaker flexibly supports multiple modes such as active-passive, N+1, and N-N. By installing the OCF agent on each node, it can determine whether another node is running Glance and Keystone services correctly, helping Pacemaker start, stop, and monitor these services.
5) High Availability of Swift Object Storage
Generally, OpenStack’s distributed object storage system, Swift, does not require additional HA configurations. This is because Swift is designed with distributed architecture (without a master node), fault tolerance, redundancy mechanisms, data recovery mechanisms, scalability, and high Availability.
6) High Availability of the Message Queue Service (RabbitMQ)
If RabbitMQ fails, messages may be lost. Multiple HA mechanisms can be used:
The publisher confirms method notifies when data is written to disk in case of failure.
The multi-node cluster mechanism can be used, but node failures may still lead to queue failures.
The active-passive mode ensures failover in case of failure, but starting the backup node may involve delays or even failures.
For disaster recovery and availability, RabbitMQ provides persistent queues that store unprocessed messages on disk if the queue service crashes. To prevent message loss due to delays between sending and writing messages, RabbitMQ introduces the Publisher Confirm mechanism to ensure that messages are genuinely written to disk.
For cluster support, RabbitMQ offers Active/Passive and Active/Active modes. For example, in Active/Passive mode, when a node fails, the passive node is immediately activated and quickly replaces the failed active node to continue handling message delivery.
Building a High-availability OpenStack
In general, high availability is achieved by establishing redundancy and backups. Common strategies include: Cluster mode. Multi-machine mutual backup, where each instance is backed up multiple times without a central node. Examples include the distributed object storage system Swift and the multi-host mode of nova-network.
Autonomous mode. In some cases, SPoF can be addressed by allowing each node to operate independently, eliminating master-slave relationships to reduce the impact of a failed control node. For instance, nova-api is only responsible for its own node.
Active-passive mode. This common model involves a passive node that remains in a standby and backup state, switching over when a failure occurs. Examples include MySQL high-availability clusters, and services like Glance and Keystone, which achieve redundancy through Pacemaker and Heartbeat.
Active-active mode. In this mode, nodes back up and support each other. For example, RabbitMQ uses an active-active high-availability cluster, where nodes in the cluster can replicate queues. From an architectural perspective, this approach eliminates concerns about passive nodes failing to start or experiencing excessive delays.
In summary, OpenStack optimization and improvements are continuously evolving, and its deployment and applications are being explored and developed further. Practical tuning is essential—good designs and ideas need real-world testing to be validated.
Enhanced OpenStack Protection
Although OpenStack reduces the risk of single point failure in the system through redundancy and high availability architecture, data protection is still critical. To ensure business continuity, a professional backup solution must be adopted to prevent data loss and the impact of catastrophic failures.
Vinchin Backup & Recovery is a robust OpenStack backup solution offering agentless, efficient backups with features like data deduplication, compression, incremental backups, file-level-recovery and cloud archive, etc. It ensures quick recovery, seamless integration with OpenStack, and strong data security, making it an ideal choice for managing and protecting cloud environments.
Besides, data encryption and ransomware protection offer you dual insurance to protect your OpenStack VM backups. You can also simply migrate data from an OpenStack host to another virtual platform (like VMware, Hyper-V, Proxmox, XenServer, oVirt, AWS EC2...) and vice versa.
Backing up OpenStack VM with Vinchin Backup & Recovery ony requires the following 4 sreps:
1. Select the backup object.
2. Select backup destination.
3. Configure backup strategies.
4. Review and submit the job.
Vinchin Backup & Recovery is trusted by thousands of companies. Start your 60-day full-featured trial today!
Openstack High Availability FAQs
1. What is the availability zone in OpenStack?
In OpenStack, an Availability Zone (AZ) is a logical partition within a cloud environment that helps distribute workloads for better fault tolerance and resource management. Each AZ is an isolated failure domain, meaning instances within different AZs are less likely to be affected by the same hardware or network failures. AZs are typically used to group compute nodes, while storage and networking services can also have their own zones for enhanced resilience and redundancy. Users can specify an AZ when launching instances to optimize performance and reliability.
2. Is OpenStack a SAAS or PaaS?
OpenStack is primarily an IaaS platform. It provides virtualized compute, storage, and networking resources, allowing users to deploy and manage cloud infrastructure. While OpenStack can support PaaS solutions by running services like Kubernetes or Cloud Foundry on top, its core function is to deliver IaaS rather than SaaS or PaaS.
Conclusion
Ensuring high availability in OpenStack requires a combination of architectural strategies and technical solutions to eliminate single points of failure and enhance system resilience. By implementing active-passive and active-active modes, leveraging clustering techniques, and optimizing key components such as Nova, Glance, Keystone, and RabbitMQ, OpenStack can achieve a fault-tolerant and highly reliable cloud infrastructure. Additionally, distributed storage solutions like Swift inherently provide redundancy and resilience.
