Cluster

If you have a high load on your system or need increased resilience, consider running the Axon Ivy Engine in a Cluster (Axon Ivy Engine Enterprise Edition). A Cluster setup has two major advantages:

  • Performance and scalability: An Axon Ivy Engine Enterprise Edition can serve more clients than Axon Ivy Engine Standard Edition. If the number of your clients increases, you can add another Engine node to your Axon Ivy Engine Cluster.

  • High availability: In an Axon Ivy Engine Enterprise Edition installation, a single node may crash without affecting the other nodes that are still serving clients. However, if you require high availability of your Axon Ivy Engine, you also need to ensure that all other components the engine depends on (Load Balancer, Reverse Proxy/Web Application Firewall, Database Server, File Share, Elasticsearch server) provide high availability.

Compared to a single node setup, the cluster is a more complex system, has higher hardware costs and license fees: You need an Enterprise License.

The diagram below shows an Axon Ivy Engine Cluster setup. A user accesses the Axon Ivy Engine Cluster via a load balancer that routes the traffic to the nodes of the cluster. All cluster nodes share the same system database and Elasticsearch server.

digraph overview { layout=neato; node [shape=box, height="0.8", width="2", color="#007095", style=filled, fixedsize="true", margin=0, fontname="Roboto,Helvetica Neue,Arial,sans-serif", fontcolor="white", fontsize=15 ]; /* nodes */ browser [ label="Browser", pos = "0.00,0!", width="1",height="0.8",fixedsize="true"]; loadbalancer [ label="Load Balancer", pos = "2.25,0!",width="1.5",height="0.8",fixedsize="true"]; /* this is a kludge, should be a subgraph but won't display as we'd like, so create the "subgraph" ourselves by positioning the elements */ cluster [ label="Axon Ivy Cluster", pos="5.4, 0.0!", height="3.5", width="1.8", style="dashed", fontcolor="black", labelloc="t", fontsize=17]; engine1 [ label="Node 1", pos="5.4, 1.0!", height="0.8", width="1.5", color="#C7D426", fontcolor="black"]; engine2 [ label="Node 2", pos="5.4, 0.0!", height="0.8", width="1.5", color="#C7D426", fontcolor="black"]; enginespacer [label="...", pos="5.4,-0.6!", height="0.8", width="1.5", shape=none, fillcolor=none, fontcolor=black, fontsize=11]; enginen [ label="Node n", pos="5.4,-1.2!", height="0.8", width="1.5", color="#C7D426", fontcolor="black"]; systemdb [ label="System\nDatabase", pos="8,0.8!", height="0.8", width="1.5"] elasticsearch [ label="Elasticsearch", pos="8,-1.0!", height="0.8", width="1.5"] /* edges */ browser -> loadbalancer; loadbalancer -> cluster; cluster -> systemdb; cluster -> elasticsearch; /* edge labels */ browserloadbalancer [ label="HTTPS\n \n ", pos = "1,0!", height="0.8", width="1.5", shape=none, fillcolor=none, fontcolor=black, fontsize=11]; loadbalancercluster [ label="HTTP\n \nHTTPS", pos = "3.7,0!", height="0.8", width="1.5", shape=none, fillcolor=none, fontcolor=black, fontsize=11]; clustersystemdb [ label="JDBC" pos = "6.8,0.55!", height="0.8", width="1.5", shape=none, fillcolor=none, fontcolor=black, fontsize=11]; clusterelasticsearch [ label="HTTP\n \nHTTPS", pos = "6.8,-0.58!", height="0.8", width="1.5", shape=none, fillcolor=none,fontcolor=black, fontsize=11]; }

Runtime

We strongly recommend operating an Axon Ivy Engine Cluster in a containerized environment like Docker, Kubernetes, or OpenShift. Build and run your container image containing all your projects and configurations.

See our example configurations on GitHub for nginx and HAProxy on how to set up an Axon Ivy Engine Cluster with docker-compose.

Restrictions

Please consider The following restrictions if you want to run an Axon Ivy Engine Cluster:

  1. Each node needs access to all other nodes via UDP/Multicast.

  2. Each node has to use the same system database.

  3. Each node has to use the same external Elasticsearch server (or cluster).

  4. Each node has to use the same service configurations.

  5. Configuration changes are only applied to the local cluster node where you made the change.

  6. Deployment during runtime is not supported and does not work.

You can solve most of the restrictions by using a containerized environment with your own container image.

Usually, UDP/Multicast works in a single subnet only. Modern routers can overcome this restriction if you configure them accordingly. If your environment does not support this setup, you can change the cluster communication to TCP/IP. TCP/IP will induce significantly higher cluster communications overhead. Contact Axon Ivy for instructions regarding this setup.