Goal

Learn how to pre-process raster data using the Tiling Engine API to convert it into a multi-leveled tile pyramid. This can significantly improve the visualization performance.

An alternative to using the API is using the Data Connectivity Manager application which comes with LuciadFusion.

For integration in scripts, you can use the samples.fusion.engine.format.Fuser sample. This sample allows you to pre-process data using the command line.

Terminology

The Tiling Engine uses the following concepts:

  • Tile store: a tile store is the container where the pre-processed results are stored, and which contains all the metadata about the original data and the pre-processed data.

  • Asset (metadata): the input data sets are called assets, and the metadata about those data sets that is stored in the tile store is called the asset metadata.

  • Coverage (metadata): a coverage is the result of the pre-processing. It contains the produced tiles. The coverage metadata contains information about which assets were pre-processed, and about the tile structure of the resulting coverage.

Pre-process a raster data set

Before we can start the pre-processing of the data, we need to create some environments.

//Create the environments
//These are expensive resources.
//It is best to only create a single instance of each in your application
ALfnEnvironment environment = ALfnEnvironment.newInstance();
ALfnEngineEnvironment engineEnvironment = ALfnEngineEnvironment.newInstance(environment);
ALfnClientEnvironment clientEnvironment = ALfnClientEnvironment.newInstance(environment);

Once we have the environments, we first create our tile store.

We use the following code which will create the tile store if it does not yet exist.

/**
 * Returns the tilestore.
 * When not yet available, this method will first create it.
 */
private static ALfnTileStore getTileStore(ALfnClientEnvironment aClientEnvironment) throws IOException, TLfnServiceException {
  Path tileStorePath = Paths.get(System.getProperty("java.io.tmpdir"), "tilestore");
  URI tileStoreUri = tileStorePath.toUri();

  TLfnClientFactory clientFactory = new TLfnClientFactory(aClientEnvironment);
  TLfnTileStoreProvider tileStoreProvider = new TLfnTileStoreProvider(clientFactory, aClientEnvironment.getEnvironment());
  ALfnTileStore tileStore;
  try {
    tileStore = tileStoreProvider.getTileStore(tileStoreUri);
  } catch (FileNotFoundException e) {
    // When the Tile Store does not yet exists, we create one from scratch.
    TLfnTileStoreUtil.createTileStore(tileStorePath.toFile(), aClientEnvironment.getEnvironment());
    tileStore = tileStoreProvider.getTileStore(tileStoreUri);
  }
  return tileStore;
}

Now that we have a tile store, we need to:

  • Select our input data

  • Create an asset for our input data and store that in the tile-store

  • Select that asset and use it for our coverage

  • Start the actual pre-processing for our coverage

Creating the asset

The Tiling Engine knows how to create assets for the supported formats. As such, creating an asset is as simple as calling the createAsset method: 

String rasterDataToFuse = "Data/Dted/Alps/dmed";

// Create an asset.
TLfnCompositeFormat compositeFormat = engineEnvironment.getCompositeFormat();
ALfnAssetMetadata asset = compositeFormat.createAsset(rasterDataToFuse, tileStore.query(ASSET));

// Put the asset in the Tile Store so it can be fused later on.
asset = tileStore.putResourceMetadata(asset);

Once the asset is created we store it in the tile store so that we can use it when creating our coverage.

Creating the coverage

Now that the asset is available in the tile store, we use it to create a coverage:

// Create a new coverage containing the previously created asset.
TLfnRasterCoverageMetadata coverageMetadata =
    TLfnRasterCoverageMetadata.newBuilder()
                              .addAssetInfo(asset, STATUS_INCOMPLETE)
                              .id("munchen")
                              .name("Munchen")
                              .type(ELfnDataType.IMAGE)
                              .build();

// Put the coverage on the Tile Store so it can be fused later on.
coverageMetadata = tileStore.putRasterCoverageMetadata(coverageMetadata);

While the coverage is created and stored in the tile store, it does not yet contain the pre-processed tiles.

Pre-processing the data to create the tiles

All required information is now available in the tile store. To start the actual pre-processing, we need to create an ALfnEngine for our coverage, and tell the engine to pre-process the data: 

// Create the engine and start fusing the data.
TLfnEngineFactory engineFactory = new TLfnEngineFactory(engineEnvironment);
ALfnEngine engine = engineFactory.createEngine(tileStore, coverageMetadata.getId());

// Start asynchronous fusion.
Future<Void> future = engine.fuse(null);

In this example, we used an asynchronous call to start the pre-processing. This allows us to monitor the progress of the pre-processing: 

// Report progress.
Timer progressReporter = new Timer();
progressReporter.scheduleAtFixedRate(new TimerTask() {
  @Override
  public void run() {
    TLfnProgress progress = engine.getProgress();
    System.out.println("Percentage done: " + progress.getAsFraction() * 100 + "%");
    if (engine.getStatus() == ELfnStatus.COMPLETED || engine.getStatus() == ELfnStatus.FAILED) {
      progressReporter.cancel();
    }
  }
}, 0, 500);

Full code 

import static com.luciad.fusion.tilestore.ELfnResourceType.ASSET;
import static com.luciad.fusion.tilestore.metadata.ALfnCoverageMetadata.AssetInfo.STATUS_INCOMPLETE;

import java.io.FileNotFoundException;
import java.io.IOException;
import java.net.URI;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.Timer;
import java.util.TimerTask;
import java.util.concurrent.Future;

import com.luciad.fusion.client.ALfnClientEnvironment;
import com.luciad.fusion.client.TLfnClientFactory;
import com.luciad.fusion.core.ALfnEnvironment;
import com.luciad.fusion.engine.ALfnEngine;
import com.luciad.fusion.engine.ALfnEngineEnvironment;
import com.luciad.fusion.engine.TLfnEngineFactory;
import com.luciad.fusion.engine.format.TLfnCompositeFormat;
import com.luciad.fusion.tilestore.ALfnTileStore;
import com.luciad.fusion.tilestore.ELfnDataType;
import com.luciad.fusion.tilestore.TLfnServiceException;
import com.luciad.fusion.tilestore.TLfnTileStoreProvider;
import com.luciad.fusion.tilestore.TLfnTileStoreUtil;
import com.luciad.fusion.tilestore.metadata.ALfnAssetMetadata;
import com.luciad.fusion.tilestore.metadata.TLfnRasterCoverageMetadata;
import com.luciad.fusion.util.ELfnStatus;
import com.luciad.fusion.util.TLfnProgress;

public class FuseRasterTutorial {
  public static void main(String[] aArgs) throws Exception {
    //Create the environments
    //These are expensive resources.
    //It is best to only create a single instance of each in your application
    ALfnEnvironment environment = ALfnEnvironment.newInstance();
    ALfnEngineEnvironment engineEnvironment = ALfnEngineEnvironment.newInstance(environment);
    ALfnClientEnvironment clientEnvironment = ALfnClientEnvironment.newInstance(environment);

    //Obtain a reference to the tile store
    ALfnTileStore tileStore = getTileStore(clientEnvironment);

    String rasterDataToFuse = "Data/Dted/Alps/dmed";

    // Create an asset.
    TLfnCompositeFormat compositeFormat = engineEnvironment.getCompositeFormat();
    ALfnAssetMetadata asset = compositeFormat.createAsset(rasterDataToFuse, tileStore.query(ASSET));

    // Put the asset in the Tile Store so it can be fused later on.
    asset = tileStore.putResourceMetadata(asset);

    // Create a new coverage containing the previously created asset.
    TLfnRasterCoverageMetadata coverageMetadata =
        TLfnRasterCoverageMetadata.newBuilder()
                                  .addAssetInfo(asset, STATUS_INCOMPLETE)
                                  .id("munchen")
                                  .name("Munchen")
                                  .type(ELfnDataType.IMAGE)
                                  .build();

    // Put the coverage on the Tile Store so it can be fused later on.
    coverageMetadata = tileStore.putRasterCoverageMetadata(coverageMetadata);

    // Create the engine and start fusing the data.
    TLfnEngineFactory engineFactory = new TLfnEngineFactory(engineEnvironment);
    ALfnEngine engine = engineFactory.createEngine(tileStore, coverageMetadata.getId());

    // Start asynchronous fusion.
    Future<Void> future = engine.fuse(null);

    // Report progress.
    Timer progressReporter = new Timer();
    progressReporter.scheduleAtFixedRate(new TimerTask() {
      @Override
      public void run() {
        TLfnProgress progress = engine.getProgress();
        System.out.println("Percentage done: " + progress.getAsFraction() * 100 + "%");
        if (engine.getStatus() == ELfnStatus.COMPLETED || engine.getStatus() == ELfnStatus.FAILED) {
          progressReporter.cancel();
        }
      }
    }, 0, 500);

    // Make sure the program does not exit before the fusion is done.
    future.get();
  }


  /**
   * Returns the tilestore.
   * When not yet available, this method will first create it.
   */
  private static ALfnTileStore getTileStore(ALfnClientEnvironment aClientEnvironment) throws IOException, TLfnServiceException {
    Path tileStorePath = Paths.get(System.getProperty("java.io.tmpdir"), "tilestore");
    URI tileStoreUri = tileStorePath.toUri();

    TLfnClientFactory clientFactory = new TLfnClientFactory(aClientEnvironment);
    TLfnTileStoreProvider tileStoreProvider = new TLfnTileStoreProvider(clientFactory, aClientEnvironment.getEnvironment());
    ALfnTileStore tileStore;
    try {
      tileStore = tileStoreProvider.getTileStore(tileStoreUri);
    } catch (FileNotFoundException e) {
      // When the Tile Store does not yet exists, we create one from scratch.
      TLfnTileStoreUtil.createTileStore(tileStorePath.toFile(), aClientEnvironment.getEnvironment());
      tileStore = tileStoreProvider.getTileStore(tileStoreUri);
    }
    return tileStore;
  }
}