Geospatial data management and publication in LuciadFusion Studio

If you want to publish fresh data as quickly as possible, without any custom styling, data integration, or any other kind of extra packaging, you can simply add a single file containing the data to LuciadFusion Studio, and create a service for it.

To quick-publish your data:

You can now access the data from an appropriate service client by pointing the client to the service endpoint URL.

You can add your folders or tables of geospatial data to LuciadFusion Studio by identifying them as data roots. LuciadFusion Studio also allows you to directly add your geospatial data by identifying a data source, such as a file.

Alternatively, you can upload a dataset.

From there, you can immediately proceed to data publication on a LuciadFusion web service, or you can bundle the data with other data and styles before you publish it.

LuciadFusion web services can serve geospatial data to potentially thousands of operators through the following services:

LuciadFusion accepts and supports most data types without pre-processing or loss of information: vector data, raster data, mesh data, tiled multi-level data such as LuciadFusion coverages, and data with multiple bands or dimensions, such as satellite imagery and weather data. In addition, LuciadFusion accepts raw point cloud data, automatically pre-processing it into a spatially indexed multi-level file database.

To add a data root, you need to tell LuciadFusion Studio to crawl a data location.

To add a data root to LuciadFusion Studio:

To track crawling progress, go to the JOBS tab. The top entry on the JOBS tab shows the status and a progress bar for the crawling job you started most recently.

LuciadFusion Studio displays a browser notification when it has finished crawling the data location.

To view details about the files encountered during the crawl, open the job details screen by clicking the link provided in the PATH column on the JOBS tab. For more information, see Viewing crawled file details.

Aside from the data roots you define yourself, there also is a system data root. File servers and databases are user-defined data roots, while data you upload to LuciadFusion is stored in the system data root.

Instead of adding the location of your data as a data root, you can choose to just import the files with geospatial data into LuciadFusion. LuciadFusion adds the data in the files to an internal data root, and makes the geospatial information searchable by crawling the metadata that comes with the data.

You can upload data by uploading separate files, or by uploading a compressed zip file. Studio unzips all zip files automatically and stores the content as individual data entries. As a result, you can use zip files to upload folder structures, if that is required by your data format.

Instead of using a data root, you can also add your data directly to Studio. Adding data directly means that you point LuciadFusion to a file instead of a directory or database. That file may be an entry point file for an entire data set consisting of many supporting files. For instance, an OGC 3D Tiles dataset typically has a tileset.json file as the entry point for the supporting files.

The distinction between adding data directly and uploading files is that you don’t store the files in a local data root when you add them directly. The file is left in place. The distinction with a data root is that LuciadFusion just interprets the file you are adding, and does not go on to discover and index all the underlying files. The data in the underlying files is accessible through the entry point file from LuciadFusion Studio.

To add data directly:

You can also add styles directly by going to the STYLES tab, and clicking the ADD STYLE button.

The DATA tab is your starting point for finding and inspecting data, adding data to products, publishing data as a service, and removing data.

To quickly retrieve a data set from the list, you can use the Search bar on top of the data list. For more information about how you can efficiently use search, see Appendix D, LuciadFusion Studio search capabilities.

To preview a data set on a map, select a line in the list of data.The preview and basic information about the data are displayed to the left of the list. Raster data will be previewed automatically but the server may only render its bounds with a hash fill if the image density is too high, in which case you can simply zoom in to see the actual image. See the related developer article for more. For vector data, only the bounds will be added to the map by default and you will be asked to click on the map to load the vector data if you wish to preview it. This is to prevent overloading the server with drawing a lot of vector data at a small scale. All other data, such as point cloud and 3D mesh data, cannot be previewed and only the bounds will be added to the map.

To see all the available details about a data set, click the title in its data list entry. From the data detail view, you can proceed straight to data publication, or import and download metadata files.

If data has changed on disk, you can refresh the data in LuciadFusion Studio to see the updates.

To refresh data:

You can also refresh styles by going to the STYLES tab, clicking the title of a style and clicking the REFRESH button.

To remove data in LuciadFusion Studio:

To remove crawled data from LuciadFusion Studio, you can configure crawl jobs to detect when files are removed from a data root. As a result, you can remove the files from disk first, and make LuciadFusion Studio automatically remove its corresponding data. See Detecting deleted files during crawling for more information.

You can choose to download metadata about a particular data set, or upload metadata for that data set. The only prerequisite is that imported metadata must comply with the ISO 19115 specification.

To obtain the metadata captured about a particular data set in an XML file:

To upload a metadata file with additional information about a data set:

You can determine how your data will look on the map by adding OGC SLD/SE style files.

The easiest way to change the styling of particular datasets is to upload an OGC SLD/SE style file to LuciadFusion Studio. Then, create a product that contains both the data sets and the style file.

To switch styling according to target service or audience, you can create several products, each of which contains the same data set but a distinct style file.

From the available data, you can integrate data sets and style files into products.

The Products tab allows you to manage your products. It displays all the products created so far in a list.

To quickly retrieve a product from the list, you can use the Search bar on top of the product list. For more information about how you can efficiently use search, see Appendix D, LuciadFusion Studio search capabilities.

To see a product preview and basic product details, select the product from the list. The map preview and product details are displayed on the left. The restrictions described for previewing data also apply to previewing products, see Managing your data for details.

To see all product details and the data added to the product, click the title of the product list entry.

The UPLOAD button on the PRODUCTS tab allows you to add more data and style files to LuciadFusion Studio.

The SERVICES tab allows you to manage all services from an overview. You can search for services, create and delete them, and start or stop them from the list of services.

The Studio Catalogue Service is permanently listed in the SERVICES overview. You can stop the catalog service, but you cannot delete it. You cannot create a second catalog service either.

To quickly retrieve a service from the list, you can use the Search bar on top of the service list. For more information about how you can efficiently use search, see Appendix D, LuciadFusion Studio search capabilities.

You can follow several routes to publish your data on services. Both the DATA and the PRODUCT tabs offer options to publish data in new services:

In both cases, you can proceed to add the necessary service information on the Create service dialog box. See Creating a new service for more information.

To add data to an existing service, you first need to package it as a product. Next, you can add it to a service in two ways:

If you want to swap outdated data with new data in a running service, you can do so without any action required on the client side. Service clients can maintain their connection to the same layer ID served by the same service URL as before.

To swap data in a running service:

LuciadFusion Studio allows you to regularly re-crawl data roots for changes.

Once you have created a data root, LuciadFusion Studio sets up and executes a one-time crawling job to discover and register the data. That job is your starting point for re-running a crawling job, and for the future scheduling of automatic crawling jobs. You can choose to set up a simple crawling schedule based on frequency, or a more complex schedule based on a Cron expression.

To view details of how each file was handled during a crawl job, you can consult the details page of the crawl job.

The crawl job DETAILS section shows general file statistics for the job. The file statistics show how many files were crawled successfully, how many were skipped, and how many files failed to process.

The possible status values are:

On the right, there is a searchable file list. The file list shows the file name, path, and status. You can filter the list by status by selecting a specific status from the first drop-down menu.

To show only those files added during the last run of the crawl job, select Last Run Only from the second drop-down menu.

OSGB 3D meshes and LIDAR point clouds are typically served as OGC 3D Tiles, an efficient multi-leveled data structure for those types of files. Similarly, elevation data is served as a more efficient LuciadFusion coverage. Turning mesh and point cloud data into OGC 3D Tiles or elevation data into a coverage may take LuciadFusion Studio a while, depending on the size of the data sets.

To monitor the progress of those kinds of jobs, go to the Preprocessing jobs section in the lower half of the JOBS tab.

If you want to pause or stop the job, click the Stop/Pause button further on the right.

To see more job details, click the service name in the SERVICE column. You can also re-start and stop the pre-processing job from the job details screen.

You can change the background imagery in the map previews displayed on the DATA, STYLES, PRODUCTS, and SERVICES tabs.

To restore the original LuciadFusion Studio background image for the map preview:

You can monitor the log output produced by the LuciadFusion Platform server.

Go to the SETTINGS tab, and select LOGS from the side navigation bar.

The LOGS section allows you to scroll through the most recent LuciadFusion server logs stored in the server log file. For the full server log, go to the logs/fusion.log file in your LuciadFusion installation folder.

The LuciadFusion Studio allows you to monitor how much main memory, heap and non-heap memory is consumed by the LuciadFusion server.

To check memory usage, go to the SETTINGS tab, and select STATUS from the side navigation bar.

The Memory graph allows you to monitor how much how much server memory there is, and how much of it is still available.

The Heap graph displays the total size of the heap, which is memory used for the storage of Java objects. The graph lines show how much of the heap memory is committed, initialized and used by the JVM.

The Nonheap graph displays the total size of non-heap memory, which is memory used for other purposes than object storage. The graph lines show how much non-heap memory is committed, initialized and used by the JVM.

To prepare the point cloud data for visualization by LuciadFusion Studio, and later on by visualization clients, you need to make sure that georeference information is available for the data. If LuciadFusion can’t detect georeference information, it won’t make the data available in Studio for serving.

Once georeference information is available for all the LAZ data in the folder, go to SETTINGS > DATA ROOTS > ADD DATA ROOT, and select the folder with the LAZ files and the georeference files.

LuciadFusion starts the crawling process, and adds all the LAZ files to the DATA tab.

You can set up LuciadFusion so that it regularly re-crawls the data root folder for new data, and adds that data to LuciadFusion Studio. For more information, see Scheduling and monitoring automatic crawling.

We want to publish our point cloud data as an OGC 3D Tiles service. This is the designated service type for point cloud data and 3D mesh data. It preserves all point property information for full styling and filtering flexibility at the client side. Therefore, there is no need to combine the point cloud data with other, less suitable data types, nor do we need to add styling information at the product level. We can go straight to the creation of the service, and add all the LAZ files at once. LuciadFusion will create a product containing all the data in the background.

At this point, LuciadFusion Studio starts tiling and multi-leveling the point cloud data. While it is taking care of this pre-processing job, your new service will have the PENDING status.

OGC 3D Tiles is also the designated service type for 3D mesh data. In this scenario, we set up a service for Binz mesh data.

At this point, LuciadFusion Studio starts tiling and multi-leveling the Binz data. While it is taking care of this pre-processing job, your new service will have the PENDING status.

You are now ready to use the data in client applications, such as LuciadLightspeed and LuciadRIA applications.

To connect to an OGC3D Tiles service and visualize the data, you need to know the endpoint URL of the service.

To find out what the endpoint URL is, go to SERVICES and click the title of the OGC 3D Tiles service in the list to open the details screen. Find the ENDPOINT URL entry.

You can copy that URL to the data loading code for your application, or fill it in the Connect to dialog in Lucy, for instance.

To set up the data root, go to SETTINGS > DATA ROOTS > ADD DATA ROOT, and select the folder with the DHM files.

LuciadFusion starts the crawling process, and adds all the DHM files to the DATA tab.

An LTS service offers one coverage for each product it contains, so we need to create a product with the DHM elevation files first. Combining multiple individual elevation data files in a single product allows you to serve this data as a single coverage through an LTS service. If you want to offer multiple coverages, you should create a product for each coverage you want to offer.

To create the product:

You can now create an LTS service with the product to start serving the elevation data in the product.

As mentioned earlier, an LTS service serves a coverage for each product that it contains. A product in an LTS service must contain either a single LuciadFusion coverage, or one or more elevation data files. A product containing a LuciadFusion coverage can be served directly in an LTS service. On the other hand, products containing elevation data are automatically pre-processed into a LuciadFusion coverage when you use them to create an LTS service, or when you add them to an LTS service. This is the case for the product with DHM data.

To create an LTS service for the product with DHM data:

At this point, LuciadFusion Studio starts generating a Fusion coverage. While it is taking care of this pre-processing job, your new service will have the PENDING status.

The pre-processing may take a while if you loaded a large elevation data set. To track the status of the pre-processing job, you can go to the JOBS screen. It shows the running pre-processing job in the lower half of the screen. You can click the job entry for more details.

As soon as the pre-processing job is done, the LTS service status switches from PENDING to RUNNING.

Once the LTS service is running, you can connect to it from a LuciadRIA application.

To connect to an LTS service from LuciadRIA, you need to know the endpoint URL of the service, the coverage ID, and the coverage parameters.

The leftmost search bar allows you to perform a free-form search through the most important metadata fields of your imported data, styles, products or services, depending on the page where your search takes place. The tables below list the fields that are included in the free-form search per tab.

The drop-down menu right next to the free-form search bar is automatically filled with all data types present in the data that is being crawled.

The two rightmost search bars allow you to specify a search range with a start and end date. All data entries with an import date or time bounds between or equal to these limits will be shown.