A basic multi-node setup

In this guide, we walk you through configuring and running a multi-node setup of LuciadFusion instances behind a reverse proxy. We create an example setup with just two instances, but you can add more if necessary.

To keep things simple for our example, we use LuciadFusion development installations that run on the same machine. We also mention how to use deployment versions of LuciadFusion, though. It’s also straightforward to move the setup from a single machine to more than one machine.

The example setup starts from new installations of LuciadFusion. If you want to transform an existing single-node setup to a multi-node setup, go over How to transform single-node setup into multi-node LuciadFusion setup before reading this guide.

This guide doesn’t discuss how to set up parts of LuciadFusion as microservices, or how to use LuciadFusion with a service orchestration framework like Docker, Kubernetes, or Helm.

Contact our support desk services at support.luciad.gsp@hexagon.com with your use case if you want LuciadFusion support for those setups.

Getting started

Requirements

To create a LuciadFusion multi-node setup, you need:

  • The installer of LuciadFusion 2020.1 or later.

  • A reverse proxy. We use an Apache HTTP Server for our example setup, but configuring other reverse proxies with the provided instructions should be straightforward. A possible alternative is NGINX, or a cloud-based solution like Amazon ELB with automatic scaling.

  • A PostgreSQL, Oracle or SQL Server database to use as the shared Studio database. We use a PostgreSQL database but any of those three databases works. Note that this setup doesn’t support H2 databases.
    You can host the database on cloud services, with Amazon RDS for example.

Running LuciadFusion in a multi-node setup requires a license for each machine that LuciadFusion runs on. Please contact customerservices.luciad.gsp@hexagon.com for more information about the LuciadFusion license options.

Installing more nodes

The LuciadFusion installation is the same for one node and several nodes. Just follow the instructions from the installer.

For the example setup we run the installer twice, to install LuciadFusion in two locations on the same computer:

  • One installation at home/fusion/node1. We refer to this instance as node1.

  • One installation at home/fusion/node2. We refer to this instance as node2.

If you are using the deployment version of LuciadFusion, you can use a single install to build a deployment version of LuciadFusion for each instance you need.

If you use more than one machine in your multi-node setup, you can either install LuciadFusion on each node, or generate a deployment version for each node from a single installation.

You can only use instances with the same version of LuciadFusion together in a multi-node setup, even when you are using a patch release.

Configuring LuciadFusion to work in a multi-node setup

After you installed LuciadFusion, you need to change the configuration files of LuciadFusion to enable a multi-node setup. This includes turning off indexing, configuring the shared database, and setting a datastore path.

Turn off indexing

First, you must turn off indexing for searches in LuciadFusion Studio.

To turn off search indexing, set the fusion.studio.configuration.textIndexingEnabled to false in the fusion.common.yml configuration file. You can find the fusion.common.yml configuration file in the config folder of the LuciadFusion installation directory.

For our example setup, we turn off indexing for the two LuciadFusion installations. The property in the YML files looks like this: 

fusion.studio.configuration:
    textIndexingEnabled: false

If you can’t find the textIndexingEnabled property in the fusion.common.yml configuration file, make sure that you are using LuciadFusion 2020.1 or later.

The reason why you need to turn this off is that the datastore used for indexing doesn’t support concurrent access from more than one instance. LuciadFusion only uses the index for searches in the Studio web application, so its impact on the use of LuciadFusion is limited.

Searches in the Studio web application fall back to simple database searches when you turn off indexing. Advanced Lucene searches no longer work, and regular searches might be slower because of that.

Don’t turn indexing back on after using LuciadFusion Studio with indexing off, because that might cause LuciadFusion to return incorrect results.

LuciadFusion builds up the index while you create, change, and remove products and services. This means that the index might not reflect the actual state of the data if you turned it off for some time.

Use the same database

Next, we configure each LuciadFusion instance to use the same database instance.

Open the application-fusion.development.yml file if you are using the development version of LuciadFusion. If you are using the deployment version, open your production YML file. Look for the fusion.studio.db property. Follow the instructions for creating a database below the property.

For our example setup, we create a PostgreSQL database fusion_studio_multi_node on the same machine as the LuciadFusion instances.

Once you created the database, change the url, username, and password properties in the YML files to match.

For the example setup, we change the YML files of both the node1 and node2 installation to point to the fusion_studio_multi_node database. This configuration shows the changed properties in the YML file. We commented out the default H2 configuration, and set the PostgreSQL properties. 

fusion.studio.db:

    #url: "jdbc:h2:${fusion.studio.configuration.databasePath}/fusion_studio/core"
    #username: admin
    #password: admin

    url: jdbc:postgresql://localhost:5432/fusion_studio_multi_node
    username: fusion
    password: admin

Point to the datastore

As a final configuration step, we must set the datastore to the same location for all LuciadFusion instances. LuciadFusion uses the datastore for storing uploaded data and pre-processed data. By default, the path to the datastore is a relative path, so it’s best to change this to an absolute path. If you plan to run LuciadFusion instances on different machines, you must use a location that’s accessible to all instances, for example, a network drive.

For our example setup, we create a folder /home/fusion/datastore, and change the datastore property in the application-fusion.development.yml file of both installations to point to this folder. You can see the changed property in the configuration below. Change your production YML file if you plan to use a deployment version of LuciadFusion. 

fusion.studio.configuration:
  datastore: /home/fusion/datastore

With this, we conclude the necessary LuciadFusion configuration for a multi-node setup.

Configure ports, and other changes

At this point, you can change other configuration settings in the YML files if you need to.

If you plan to use a deployment version of LuciadFusion, you can build those now. See Build the LuciadFusion server application for deployment for instructions.

We make one more change to the configuration for our example setup. Because both instances run on the same machine, they must use a different port. We leave the port of node1 at the default value of 8081, but change node2 to use port 8082 instead. We change the fusion.common.yml configuration file — or the production YML — of node2 for that. This is the changed property: 

server.port: 8082

Running LuciadFusion in a multi-node setup

Now that we finished configuring LuciadFusion, we’re ready to start running LuciadFusion in a multi-node setup.

First start-up

When you want to start up LuciadFusion in a multi-node setup for the first time, don’t start up all instances at once.

The first time LuciadFusion starts up, it needs to go through several configuration steps, such as creating a schema in the database, and creating an uploads data root. If you start two or more instances at the same time, you may cause errors because both instances could try to execute the same step.

In our example setup, we start up node1 by running the /home/fusion/node1/FusionServer.sh script, and wait until we can access the Studio web application. Because we access Studio locally, the address is http://localhost:8081. After that, we start up node2 by running /home/fusion/node2/FusionServer.sh. If you are using a deployment version of LuciadFusion, you can start LuciadFusion by using one of the build artifacts. See Selecting the right build artifact for more information.

To test the example multi-node setup, create a WMS service called example_wms in the Studio web application of node1, and add some data to it. The endpoint URL of the service is http://localhost:8081/ogc/wms/example_wms. Because node2 is part of the same multi-node setup, you can also access it at http://localhost:8082/ogc/wms/example_wms. Test this by connecting to that URL with a WMS client, LuciadLightspeed Lucy for example.

Guidelines and Limitations

To ensure that LuciadFusion runs correctly in a multi-node setup, we recommend that you follow the guidelines in this section. The basis for these guidelines are the current technical limitations of running LuciadFusion in a multi-node setup.

Guideline 1: only use Studio on a single node

Use the Studio web application and REST API on a single node only. Running Studio on more than one node could result in inconsistent behavior for some features of LuciadFusion. Because Studio initiates pre-processing, you need to place pre-processing on the same node as Studio.

To set aside a node for Studio and pre-processing, see the setup scenario for a Designated Studio and pre-processing node.

The main reason for this guideline is that the actual execution of a job happens on a single instance of LuciadFusion in a multi-node setup, and that none of the instances in the setup knows which instance that is. This means, for example, that you cannot stop a job on an instance that’s distinct from the one where it started. In our example setup, if you start a job on node1 and then try to stop it on node2, the job continues to run on node1 because that is where the actual execution happens. Note that this happens both when you use the Studio web application or the Studio REST API.

Another limitation is that the Studio web application always has a session with a single LuciadFusion instance. Switching between instances requires you to log in again. You only see notifications on the instance that triggered the notification. For example, you can only see a notification of a finished job on the instance where the job was started.

Finally, deleting the same data root on more than one node or deleting many data roots at the same time can result in errors that prevent the deleting of the data root.

Guideline 2: make sure that no jobs are queued, running, or scheduled when you start a LuciadFusion instance that’s part of a multi-node setup

LuciadFusion resumes jobs at startup to ensure these jobs continue after a restart. A LuciadFusion instance doesn’t know if a job is already executing on a different instance in a multi-node setup. It resumes all queued, running, or scheduled jobs, even if these jobs are already queued, running or scheduled on another instance in the setup. That’s why it’s important to make sure that there are no jobs with any of these states on any of the instances in a multi-node setup when you start another instance.

You can also configure LuciadFusion to not resume jobs on startup by setting the YML property fusion.studio.configuration.resumeJobsOnStartup to false. You can usually turn this off on all nodes except the Studio node.

Upgrading to a newer version of LuciadFusion

To upgrade the LuciadFusion instances in a multi-node setup, follow the instructions in How to upgrade a multi-node LuciadFusion setup. We encourage you to read it when you are setting up a multi-node configuration so that you are aware of the steps before you actually need to upgrade.