Working with Earth tilesets

The class ALcdMultilevelImageMosaic in LuciadLightspeed 's image processing framework com.luciad.imaging is the recommended way of working with multi-leveled images. However, you can continue using the com.luciad.earth package and ILcdEarthTileSet objects for reasons of continuity and backward compatibility.

For more information about working with ALcdMultilevelImageMosaic, see How to model raster data for image processing.

Earth tilesets

A tileset is a dataset which has been prepared into a hierarchical tiling structure. In the LuciadLightspeed Earth package, it is defined by the interface ILcdEarthTileSet.

A tileset provides multiple levels of detail, each of which contains a regular grid of tiles. Level 0 is the least detailed, and each subsequent level doubles the numbers of rows and columns in the tile grid. The combined levels form a multi-resolution tile pyramid in which one tile on level N corresponds to a block of 2x2 tiles on level N+1.

hgrid
Figure 1. Detail levels in an ILcdEarthTileSet

The tile grid is defined in a 2D coordinate system, a geodetic reference or a grid reference for example, and its bounds. Each level covers the exact same geographic area, but does so using an increasingly large number of tiles. All tiles contain approximately the same amount of data, but the higher levels contain more detailed data than the lower levels.

A tileset may load tiles from a disk or some other storage medium, or it may generate tiles on the fly based on some other data source. The tileset may process requests on a separate thread of execution, so that the application does not pause or stutter while tiles are being retrieved. This is important to obtain a fluid and responsive visualization.

Although the tileset defines tiles as being laid out in a regular grid, it does not require that every cell in the grid is effectively populated. This allows tilesets to create sparse data structures, which is useful to create high-resolution insets in low-resolution base data, for instance.

The contents of the tiles are not specified by the tileset. The tileset interface does not enforce that tiles contain data of a regularly gridded nature, for example. Tiles may also contain 3D representations of geographic features, such as buildings, roads and vegetation, 2D or 3D raster data, such as weather data, or even non-visual data, for example textual descriptions of the area underlying the tile.

For a 3D terrain, each tile may contain a 3D geometric representation of the terrain elevations, derived from regularly spaced sample locations in a digital elevation model, and a 2D image to be used as the texture map for that terrain tile.

Earth tileset coverages

An ILcdEarthTileSet contains one or more data coverages. Each data coverage is represented by a ILcdEarthTileSetCoverage, and all data coverages of a single tileset share the same tiling structure. The coverages describe different sets of data that are available from the containing tileset. Therefore, when a tile is requested from an ILcdEarthTileSet it is always for one particular coverage, which is passed to the tileset as an argument. Each coverage has a name which must be unique within its containing tileset.

Each ILcdEarthTileSetCoverage has a coverage type, which is one of the following:

  • ELEVATION: the coverage contains gridded elevation data.

  • MULTIVALUED: the coverage contains multidimensional scalar values.

  • IMAGE: the coverage contains image data.

  • GEOMETRY: the coverage contains geometric primitives: points, lines, polygons, 3D meshes, and so on.

  • OTHER: the coverage contains non-visual data.

Retrieving tiles from a tileset

A tile is obtained from a tileset by invoking the produceTile() method. This method takes the following arguments:

  • The level from which a tile is desired

  • The column and row indices of the desired tile, starting from the origin of the tile grid in the lower left corner

  • The ILcdEarthTileSetCoverage from which a tile is desired

  • The geographic reference in which the resulting tile should be defined - this is explained in more detail in the next section.

  • The format in which the tile is to be returned - the format is specified as a TLcdEarthTileFormat and determines the type (java class) of the data object in the requested tile.

  • A mode that steers the asynchronous behavior of the tileset

  • A callback that is to be invoked when the tile is ready

  • A context parameter that can be used to pass application-specific data into the tileset

The callback mechanism is what allows tilesets to operate asynchronously. If a tileset supports multithreaded operation, produceTile() only schedules a request to be executed at a later time. When the tile becomes available, the callback is invoked in order to return it to its requestor. The mode parameter of produceTile() allows you to specify whether you want asynchronous behavior (TLcdEarthTileOperationMode.PREFER_ASYNCHRONOUS) or not (TLcdEarthTileOperationMode.FORCE_SYNCHRONOUS).

The ILcdEarthTileSetCallback interface contains two methods. The first, tileAvailable(), is used to notify the requestor that a tile has become available for use. The argument of this method is an ALcdEarthTile object that contains the requested tile data. The second method in the callback is tileNotAvailable(). This method is used to inform the requestor that a tile could not be produced. This mechanism allows for the creation of sparse tile grids, in which not all areas are necessarily subdivided to the same level of detail.

If a requested tile is available, it is returned in the form of an ALcdEarthTile object.

Using Earth tilesets to work with 3D terrain

A 3D terrain uses elevation data and image data as input. Both the elevation data and the image data consist of regularly spaced sample points, so that the terrain can be trivially split into tiles of equal sizes, 64x64 values per tile for the elevation and 256x256 pixels per tile for the images, for example.

The basic steps to visualize 3D terrain with the com.luciad.earth.* packages are:

  • Pre-processing

    • Collect source data: elevation models from which to derive the terrain elevations, and raster data to use as terrain images

    • Preprocess the source data to create a terrain tile repository on disk.

  • Visualization

    • Open the terrain tile repository in the application

    • Add a layer with the repository to view

A tile repository is an on-disk backing store for tiles. Once the source data has been processed into a set of tiles, the tiles are written to a tile repository, from which other tilesets can retrieve them.

These steps are explained in more detail in the following sections.

Pre-processing to generate a 3D terrain

Pre-processing work flow

The tools described below for preprocessing of 3D terrain data are available with LuciadLightspeed out of the box. However, the LuciadFusion product provides data management tools that are both more powerful and easier to use. The use of LuciadFusion to prepare 3D terrain data is particularly recommended when very large amounts of data need to be processed and/or when the data needs to be updated after the initial preprocessing step.

The general work flow when working with ILcdEarthTileSet is as follows:

tileset flow
Figure 2. Work flow when using ILcdEarthTileSet

When pre-processing 3D terrain, the work flow is extended as follows:

preprocessor flow
Figure 3. Work flow when pre-processing 3D terrain
  1. A tileset is created which produces terrain imagery and elevation tiles based on the provided source data. This tileset may perform resampling, transformation or other operations on the data which are too expensive for the application to perform on the fly.

  2. Tiles from this tileset are written into a tile repository using a codec mechanism.

  3. Another tileset can read the tiles back from the tile repository using those same codecs.

  4. From there, the application can use the tiles for visualization. The tiles are delivered in a ready-to-use format, so the application does not need to repeat the expensive processing steps performed by the original tileset.

Working with tile repositories

The central component in the terrain pre-processing work flow is a TLcdEarthTileRepository. As illustrated in the previous section, a tile repository acts as a backing store for a tileset. TLcdEarthTileRepository implements the ILcdEarthEditableTileSet interface. These methods transparently read or write data from or to the repository, thus providing a simple persistence framework for the output of a tileset. TLcdEarthTileRepository also implements the ILcdEarthIterableTileSet interface, allowing iteration over its tiles.

TLcdEarthTileRepository stores tiles in a database on disk using a codec mechanism. The high-level data structure and indexing mechanisms of this database are defined by TLcdEarthTileRepository itself, but the encoding format of the individual tiles is defined by the codecs.

You can use the TLcdEarthRepositoryModelDecoder to open an existing repository and the TLcdEarthRepositoryModelFactory to create a new repository. These classes do not only open or create the database but also read and write the repository model reference and the asset model that describes its contents.

Before a repository can be used, for each ILcdEarthTileSetCoverage, an appropriate ILcdEarthTileDataCodec needs to be registered. The tile data codec is responsible for encoding and decoding the contents of tiles in a coverage. The output of the tile data codec is what is eventually written into the repository. To register a codec, call the repository’s addTileDataCodec() method, with the name of the ILcdEarthTileSetCoverage and the codec itself as arguments. The coverage name must be unique.

In the case of a 3D terrain repository, for instance, the terrain elevation is in a coverage with TLcdEarthElevationData as its data format. In this case, a TLcdEarthElevationDataTileDataCodec can be registered for that coverage. Similarly, a TLcdEarthImageTileDataCodec or TLcdEarthSeparateAlphaImageTileDataCodec can be registered to handle the terrain image tiles, which use BufferedImage as their data exchange format. The first codec can only encode opaque terrain texture tiles while the latter also supports (partially) transparent terrain texture tiles. Other ILcdEarthTileDataCodec implementations can be found in the com.luciad.earth.repository.codec package. Custom-written codecs can also be plugged in, so that TLcdEarthTileRepository can be used with any tileset regardless of the data formats it uses.

The repository must also contain all the ILcdEarthTileSetCoverage instances from which you wish to store tiles. Data coverages can be added to the repository using the addTileSetCoverage() method. This step is only necessary the first time you want to add tiles from that coverage. When you later reopen an existing repository, the data coverages will already be there.

Once the repository has been opened and codecs have been registered, it can be used to read or write tiles. This is as easy as calling produceTile() or addTile(), respectively. The addTile() method takes an ALcdEarthTile as its argument and writes it to the repository, using the appropriate ILcdEarthTileDataCodec to serialize the tile’s data. The produceTile() method takes the ILcdEarthTileSetCoverage, the level and the row and column indices of the tile as arguments, and returns the corresponding ALcdEarthTile if it exists.

Note that in order to minimize disk access, TLcdEarthTileRepository may buffer up multiple addTile() calls. To ensure that all submitted tiles are effectively written to the disk, the application must call the commit() method. When the application finishes working with the repository, it should call the dispose() method to ensure that all resources held by the repository, such as file locks, are released.

Configuring the repository for tileset compatibility

When pre-processing data to a repository the configuration of this repository must be chosen carefully to avoid a performance penalty when combining this data later on-the-fly with other tilesets. Such a situation occurs for example when compositing raster data on-the-fly on top of a 3D terrain repository.

To ensure optimal performance in these situations all repositories must have the same general structure. It is advised to use the same structure as other Luciad products that use Earth, Lucy for example. The TLcdEarthRepositoryModelFactory can be used to create a repository that has this advised structure:

  • model reference: TLcdGeodeticReference

  • bounds: from -90°,-180° to 90°,180° . The entire earth surface, for example.

  • #levels: 24

  • #rows at level 0: 2

  • #columns at level 0: 4

Defining Earth asset metadata

When building a terrain repository, the terrain pre-processor requires some metadata about the source data used to construct the 3D terrain. Each item of source data is represented by an ILcdEarthAsset, and the combined collection of assets is stored in an ILcdModel which in turn provides the pre-processor with all the metadata it needs.

An ILcdEarthAsset has the following properties:

  • A model decoder that can be used to decode the data represented by the asset

  • A source name, which points to the data that should be decoded by the aforementioned model decoder

  • The bounds of the asset

  • The asset’s coverage type: elevation, image or geometry, as explained in Earth tileset coverages

  • The asset’s last modification data, which is used by the pre-processor to check if incremental updates to the 3D terrain can or should be performed.

The following implementations of ILcdEarthAsset are available:

An asset model can be encoded and decoded to and from disk using TLcdEarthAssetModelCodec. This class makes use of ILcdEarthAssetCodec objects to convert ILcdEarthAsset instances into a set of key/value pairs, represented by a java.util.Properties object. These pairs are then saved in an XML-based file format, and can be used to reconstruct the asset when decoding the asset model. By saving the asset model to disk, it can be used to perform incremental updates to a 3D terrain by adding assets and synchronizing an existing 3D terrain with the new version of the asset model.

Here is an overview of the steps required to supply the pre-processor with an ILcdEarthAsset model:

  • To create a new 3D terrain:

    1. Create ILcdEarthAsset objects and add them to an ILcdModel.

    2. Encode the asset model using TLcdEarthAssetModelCodec for later use.

    3. Use the asset model with the terrain pre-processor.

  • To update an existing 3D terrain:

    1. Decode an asset model using TLcdEarthAssetModelCodec.

    2. Create additional ILcdEarthAsset instances and add them to the asset model.

    3. Use the asset model with the terrain pre-processor.

Producing Earth Tilesets

This section briefly describes the ILcdEarthTileSet implementations that can be used to produce 3D terrain tiles:

The image tiles produced by these tilesets are opaque by default. This can result in visible borders if the source data does not cover the entire area of the tileset. To prevent these borders the produced images should be transparent when there is no data. This can be achieved by setting a transparent background color and image type on the TLcdEarthGXYViewTileSet.

Starting the pre-processor

The final step in generating a 3D terrain is to call the pre-processor itself. To recapitulate, the following steps should be performed before the pre-processing can start:

Pre-processing can now be initiated using TLcdEarthTileRepositoryPreprocessor. By calling the synchronizeRepository() method, this class iterates over all assets in the asset model, uses the tileset to generate the tiles corresponding to those assets, and then adds those tiles to the repository.

Furthermore, it keeps track of changes in the asset model: if an asset is added after a repository has already been created, the pre-processor will not rebuild the entire repository from scratch. Instead, it only produces those tiles that are affected by the new asset. Similarly, if the pre-processing is aborted for whatever reason and restarted at a later date, it will pick up where it left off rather than starting over.

The pre-processing progress can be monitored using an ILcdEarthPreprocessorProgressListener, which provides detailed information about what the pre-processor is doing. The progress() method in the listener is invoked after every tile that is generated. This method returns a boolean value which acts as a stop condition for the pre-processor. By having it return false, the pre-processor stops working and exits cleanly.

TLcdEarthTileRepositoryPreprocessor has the option of registering one or more ILcdEarthTileCombiner instances. An ILcdEarthTileCombiner is a class that takes four tiles, consisting of a 2x2 block of adjacent tiles, and combines them into a single new one of a lower detail level. Available tile combiners are TLcdEarthImageCombiner for terrain images and TLcdEarthElevationDataCombiner for terrain elevations. If the terrain texture tiles contain 256x256 pixel images, for instance, TLcdEarthImageCombiner takes four such images, scales them down and then combines them into a single new 256x256 pixel images.

tilecombiner
Figure 4. Operation of a tile combiner

The tile combiner mechanism exists to reduce disk access during pre-processing. Pre-processing is performed top-down, starting with the highest detail level. Because the geographic area of the tiles grows larger and larger as the detail level decreases, the tiles also cover an increasingly large number of metadata assets. Eventually, the bottom detail level will require all assets to be processed just to generate a handful of tiles. It is therefore more efficient to let the TLcdEarthMetadataTerrainTileSet handle only the topmost detail level for each asset, and to use the combiner mechanism for everything below that. Thus, once the pre-processor switches to the combiner mechanism, the source assets are no longer accessed. From this point on, each tile can be generated by combining exactly four other tiles, and because the number of tiles decreases with the level of detail, the required disk access decreases as well.

Visualizing the tile repository

To view a previously created repository, the first step is to invoke TLcdEarthRepositoryModelDecoder. This decoder creates an ILcdModel. It has a single element, which is an ILcdEarthTileSet representing the tile repository. This model can then be added to the view to visualize the repository, e.g. using TLspRasterLayerBuilder.