Skip to content

W-23431158: Link CloudHub HA/DR page to the HA/DR hub #894

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
kevintroller wants to merge 6 commits into
base: latest
Choose a base branch
from
Open

W-23431158: Link CloudHub HA/DR page to the HA/DR hub #894

Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
1c159c3
W-23431158: Link CloudHub HA/DR page to the HA/DR hub
kevintroller Jul 13, 2026
e97089b
W-23431158: Rewrite CloudHub HA/DR page to the shared skeleton with a…
kevintroller Jul 13, 2026
4eb12b9
W-23431158: Optimize CloudHub HA/DR page for AI retrieval (alt text, …
kevintroller Jul 13, 2026
1d40bae
W-23431158: Use contractions and remove 'the following' in CloudHub H…
kevintroller Jul 13, 2026
d3c2e1f
W-23431158: Use bulleted Best Practices in CloudHub HA/DR page
kevintroller Jul 13, 2026
06c6d3f
W-23431158: Remove self-referential table lead from CloudHub HA/DR page
kevintroller Jul 13, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
116 changes: 63 additions & 53 deletions cloudhub/modules/ROOT/pages/cloudhub-hadr.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,101 +4,111 @@ include::_attributes.adoc[]
endif::[]
:page-aliases: runtime-manager::cloudhub-hadr.adoc

CloudHub provides high availability (HA) and disaster recovery (DR) that protect your applications from application and hardware failures.

CloudHub provides high availability (HA) and disaster recovery for application and hardware failures.
For an overview of HA and DR across all Mule deployment models, and to compare CloudHub with other options, see xref:mule-runtime::hadr-guide.adoc[High Availability and Disaster Recovery].

CloudHub uses Amazon AWS for its cloud infrastructure, so availability depends on Amazon. CloudHub runs deployments in different regions that map to Amazon regions. If an Amazon region goes down, applications in that region become unavailable. CloudHub doesn't replicate them to other regions.

For example, when the US East region is down, the CloudHub management UI and the REST services that enable deployments remain unavailable until the region recovers. You can't deploy new applications while US East is down.

While the control plane is unavailable, the runtime plane continues to send log data and other telemetry data. The worker buffers up to 1 GB of data until the control plane recovers.

CloudHub provides persistent queues for message reliability. Within a region, persistent queues are highly available, but when the region or part of it is down—usually for a few seconds or minutes—they become inaccessible and you sometimes lose data. When the region recovers, CloudHub resumes communication with the queues.

Some CloudHub modules, such as Anypoint Object Store v1, application settings, and Insight-related information, reside in the US East region for all applications. Anypoint Object Store v2 resides in the same region as the deployed application. For both Object Store v1 and v2, when a region is down, data persists and becomes available again when the region returns to service.

Anypoint Virtual Private Cloud (Anypoint VPC) applies at the region level. When a region is down, that region's VPC is down unless you've set up a VPC instance in another region.
CloudHub runs on Amazon AWS, so its availability depends on Amazon. CloudHub maps each deployment region to an Amazon region. If an Amazon region goes down, applications in that region become unavailable, and CloudHub doesn't replicate them to other regions.

== High Availability Versus Disaster Recovery

High availability (HA) is the measure of a system's ability to remain accessible despite a system component failure. You generally implement HA by building multiple levels of fault tolerance or load balancing into a system. In CloudHub, you can achieve high availability by deploying your application with multiple workers and enabling persistent queues where appropriate.
High availability (HA) measures a system's ability to remain accessible when a component fails. You implement HA by building multiple levels of fault tolerance and load balancing into a system. On CloudHub, you achieve HA by deploying your application with multiple workers and by enabling persistent queues where appropriate.

Disaster recovery (DR) refers to the process of restoring a system to an acceptable previous state after a natural or man-made disaster, such as flooding, fires, power failures, server failures, or misconfigurations.
Disaster recovery (DR) restores a system to an acceptable previous state after a natural or man-made disaster, such as flooding, fires, power failures, server failures, or misconfigurations.

Both HA and DR increase availability, but with HA you typically see no loss of service. HA keeps the service up, DR preserves data. With DR, you usually see a brief loss of service while the DR plan runs and the system restores.
Both strategies increase availability but differ in scope: on CloudHub, HA keeps your application serving requests through worker and availability-zone failures with no loss of service, whereas DR restores service and data after a region-level disruption and usually incurs a brief loss of service while the DR plan runs.

These terms help you plan HA and DR on CloudHub:
Two measurable objectives drive your DR plan:

Recovery Time Objective (RTO):: The maximum downtime a business tolerates. RTO is the time the system takes to recover after a disruption.

Recovery Point Objective (RPO):: The maximum acceptable data loss after a disaster. RPO drives how often you back up data.
Recovery Point Objective (RPO):: The maximum data loss a business accepts after a disaster. RPO drives how often you back up data.

== High Availability in CloudHub

== Anypoint CloudHub Default Deployment Model
CloudHub builds high availability into its default deployment model, and MuleSoft manages it for you.

If the application uses multiple workers, CloudHub deploys the workers in separate availability zones by default, providing HA across availability zones. The distance between the availability zones is variable and generally doesn't exceed 350 miles.
When your application uses multiple workers, CloudHub deploys the workers in separate availability zones by default, which provides HA across availability zones. The distance between availability zones varies and generally stays under 350 miles.

image::hadr-am-web-services.png[]
image::hadr-am-web-services.png["CloudHub deploying an application's workers across separate availability zones within a single region"]

If an application uses a single worker and that availability zone goes down, CloudHub restarts the application in a different availability zone. The application can experience downtime during the restart.
When your application uses a single worker and that availability zone goes down, CloudHub restarts the application in a different availability zone. The application can experience downtime during the restart.

You can set up `status.mulesoft.com` to receive alerts when a failure occurs in an availability zone or region.
To receive alerts when a failure occurs in an availability zone or region, set up `status.mulesoft.com`.

== Shared Responsibility for Disaster Recovery
== Disaster Recovery in CloudHub

MuleSoft manages CloudHub control plane and worker infrastructure within each region. You're responsible for cross-region strategy, application-level failover, and data synchronization. This table lists who does what for disaster recovery on CloudHub.
MuleSoft manages the CloudHub control plane and worker infrastructure within each region. You own the cross-region strategy, application-level failover, and data synchronization. Disaster recovery responsibilities on CloudHub divide between MuleSoft and you:

[%header,cols="1a,2a"]
|===
|Party |Responsibility
|MuleSoft |Control plane availability: Anypoint Platform UI, deployment APIs, and platform services within the provisioned region.
|MuleSoft |Infrastructure patching, security updates, and maintenance of the worker cloud.
|MuleSoft |Multi-AZ worker distribution: when you use multiple workers, CloudHub deploys them across two or more availability zones within the same region.
|MuleSoft |Automatic restart of applications in a different availability zone when one worker or AZ fails.
|You |Define and implement a cross-region DR strategy for primary and backup regions.
|You |Decide when to trigger regional failover—for example, based on health checks or business criteria.
|You |Configure Global Server Load Balancing (GSLB) or a dedicated load balancer (DLB) and routing rules to direct traffic to a backup region during a disaster.
|You |Implement application-level failover strategy: deploy and maintain applications in more than one region when you need cross-region DR.
|You |Replicate and back up external data stores such as databases, object stores, and other systems that your applications use across regions.
|You |Set up Anypoint VPC in each region when you need network connectivity there for DR.
|MuleSoft |Keeps the control plane available, including the Anypoint Platform UI, deployment APIs, and platform services within the provisioned region.
|MuleSoft |Patches infrastructure, applies security updates, and maintains the worker cloud.
|MuleSoft |Distributes workers across two or more availability zones within the same region when you use multiple workers.
|MuleSoft |Restarts applications automatically in a different availability zone when a worker or availability zone fails.
|You |Define and implement a cross-region DR strategy for your primary and backup regions.
|You |Decide when to trigger a regional failover, for example, based on health checks or business criteria.
|You |Configure Global Server Load Balancing (GSLB) or a Dedicated Load Balancer (DLB), and set routing rules that direct traffic to a backup region during a disaster.
|You |Deploy and maintain applications in more than one region when you need cross-region DR.
|You |Replicate and back up the external data stores that your applications use across regions, such as databases and object stores.
|You |Set up Anypoint VPC in each region where you need network connectivity for DR.
|===

In short, MuleSoft keeps the platform and workers resilient within a region, and you own everything that spans regions: when to fail over, how to route traffic, which applications to run in a backup region, and how to replicate data.

=== Your Responsibilities for Disaster Recovery

If your organization needs cross-region DR, design and operate your applications for it. MuleSoft doesn't automatically replicate applications or fail over traffic to another region. You're responsible for:
If your organization needs cross-region DR, design and operate your applications for it. MuleSoft doesn't automatically replicate applications or fail over traffic to another region. You own these areas:

* Regional failover strategy: Decide when to switch traffic to a backup region—for example, after a region outage or based on health checks.
* Traffic management: Use a load balancer, cloud-based or on-premises, such as a Dedicated Load Balancer (DLB) or external GSLB, to route traffic to applications in different regions and to switch to the backup region as part of your DR plan.
* Application deployment: Deploy the same or equivalent applications in a backup region and keep them in sync with configuration and code.
* Data and state: Replicate or back up external data stores such as databases, caches, and object stores that your integrations use so applications in a DR region can access the data they need. Anypoint Object Store v1 and v2 are regional; they don't provide cross-region failover.
* *Regional failover strategy* - Decide when to switch traffic to a backup region, for example, after a region outage or based on health checks.
* *Traffic management* - Route traffic across regions with a cloud-based or on-premises load balancer, such as a Dedicated Load Balancer (DLB) or an external GSLB, and switch to the backup region as part of your DR plan.
* *Application deployment* - Deploy the same or equivalent applications in a backup region, and keep their configuration and code in sync.
* *Data and state* - Replicate or back up the external data stores that your integrations use, such as databases, caches, and object stores, so that applications in a DR region access the data they need. Anypoint Object Store v1 and v2 are regional and don't provide cross-region failover.

For guidance on designing HA and DR topologies, including active-active, warm standby, and cold standby, see xref:mule-runtime::hadr-guide.adoc[High Availability and Disaster Recovery].

=== Restoring After a Disaster

Restoration depends on the DR strategy you put in place. In general, after you confirm the primary region or application is unavailable:
Your restoration steps depend on the DR strategy you put in place. After you confirm that the primary region or application is unavailable, generally follow these steps:

. Switch traffic to the backup region.
. Switch traffic to the backup region.
+
Use your load balancer, such as a GSLB or Dedicated Load Balancer (DLB), to route traffic to the backup region. The health checks you configured earlier mark the primary as unhealthy and direct traffic to the backup endpoints. If you use cold or warm standby, bring your backup applications online.
. Bring backup applications online when you use cold or warm standby.
Use your load balancer, such as a GSLB or a Dedicated Load Balancer (DLB), to route traffic to the backup region. The health checks you configured earlier mark the primary as unhealthy and direct traffic to the backup endpoints.
. Bring backup applications online when you use cold or warm standby.
+
If the control plane is available, use Anypoint Runtime Manager or the CloudHub API to start the backup application or scale it up. If the control plane is in the same region as the failed primary, it is unavailable. You can't start or scale apps until the control plane recovers, unless you use automation that doesn't depend on the control plane.
. Verify that the backup region is serving traffic and that dependent systems use the correct endpoints or data stores.
. When the primary region recovers, optionally fail back by switching traffic from the backup region back to the primary and resyncing data when needed.
If the control plane is available, use Anypoint Runtime Manager or the CloudHub API to start or scale up the backup application. If the control plane sits in the same region as the failed primary, it is unavailable, and you cannot start or scale applications until the control plane recovers, unless you use automation that doesn't depend on the control plane.
. Verify that the backup region serves traffic and that dependent systems use the correct endpoints and data stores.
. Fail back when the primary region recovers, switching traffic from the backup region to the primary and resyncing data as needed.

Your RTO depends on how quickly you complete these steps and, for cold or warm standby, on how long the backup application takes to start or scale. For active-active setups, traffic continues on the remaining region without a switch. For more on recovery types and topologies, see xref:mule-runtime::hadr-guide.adoc[] and xref:cloudhub-2::ch2-ha-dr.adoc[].

== Considerations and Limitations

Account for these CloudHub behaviors when you plan HA and DR:

Control plane availability:: When the US East region goes down, the CloudHub management UI and the REST services that enable deployments stay unavailable until the region recovers, and you cannot deploy new applications during that time.

Log and telemetry buffering:: While the control plane is unavailable, the runtime plane keeps sending log and telemetry data, and the worker buffers up to 1 GB of data until the control plane recovers.

Persistent queues:: Within a region, persistent queues stay highly available. When the region or part of it goes down—usually for seconds or minutes—the queues become inaccessible and you can lose some data. CloudHub resumes communication with the queues after the region recovers.

Your RTO depends on how quickly you complete these steps and, for cold or warm standby, on how long it takes to start or scale the backup application. For active-active setups, traffic continues on the remaining region without a switch. For more on recovery types and topologies, see xref:mule-runtime::hadr-guide.adoc[] and xref:cloudhub-2::ch2-ha-dr.adoc[].
Object stores:: Anypoint Object Store v1, application settings, and Insight-related information reside in the US East region for all applications, and Anypoint Object Store v2 resides in the same region as the deployed application. For both versions, data persists during a region outage and becomes available again when the region returns to service. Neither version provides cross-region failover.

== Suggested Alternative Deployment Model
Anypoint VPC:: Anypoint Virtual Private Cloud (Anypoint VPC) applies at the region level. When a region goes down, that region's VPC goes down unless you set up a VPC instance in another region.

You can use a cloud-based or on-premises load balancer for applications deployed to different regions to improve your disaster recovery strategy. Configure your load balancer to perform health checks and to route traffic to your backup region when the primary region goes down. For CloudHub-specific load balancing options, see xref:dedicated-load-balancer-tutorial.adoc[CloudHub Load Balancers] and xref:cloudhub-dedicated-load-balancer.adoc[Dedicated Load Balancers].
=== Suggested Alternative Deployment Model

image::hadr-load-balancer.png[]
To strengthen your DR strategy, route traffic across regions with a cloud-based or on-premises load balancer. Configure the load balancer to run health checks and to route traffic to your backup region when the primary region goes down. For CloudHub-specific options, see xref:dedicated-load-balancer-tutorial.adoc[CloudHub Load Balancers] and xref:cloudhub-dedicated-load-balancer.adoc[Dedicated Load Balancers].

== Keep Integrations Stateless
image::hadr-load-balancer.png["A load balancer running health checks and routing traffic to a backup region when the primary region goes down"]

Keep integrations stateless. Don't share transactional information between client invocations or scheduled runs. When the middleware needs to keep data because of a system limitation, store it in an external store such as a database or messaging queue, not in the middleware infrastructure or memory.
== Best Practices

As you scale, especially in the cloud, keep each worker's state and resources independent of other workers. This model gives you better performance, scalability, and reliability.
* Keep your integrations stateless.
* Don't share transactional information between client invocations or scheduled runs.
* When the middleware must retain data because of a system limitation, store it in an external store, such as a database or a messaging queue, rather than in the middleware infrastructure or memory.
* As you scale, especially in the cloud, keep each worker's state and resources independent of the other workers, which delivers better performance, scalability, and reliability.

== See Also

Expand Down