A 3D dataset with one object selected
Figure 1. An OGC 3D Tiles dataset with custom metadata attributes

This article describes how the 3D Tiles Processing Engine processes mesh metadata, and how you can customize what metadata ends up in your data set. It also points you to the relevant mesh data handling instructions for a couple of popular mesh formats and tools.

Out-of-the-box metadata handling

As of V2020.1, the 3D Tiles Processing Engine automatically produces metadata for all mesh input files. By default, it adds specific metadata attributes only. The nature of the source data you give to the 3D Tiles Processing Engine determines which metadata types it can produce.

By default, the 3D Tiles Processing Engine can produce metadata from these sources:

The TLcd3DTilesProcessorDataTypes class reflects this default metadata model.

Vertex group ID

One source of metadata is the group ID within an OBJ file. Take this figure of a house, for example:

A diagram of a house with group ids
Figure 2. An input dataset with vertex grouping IDs

The OBJ file for this figure groups many of the vertices of the mesh together. It identifies each group of vertices with a so-called group ID.

When you look at the file, you can see the group ID:

A snippet of an OBJ file, showing the group ID before defining the faces that belong to it.
# start of the chimney group
g chimney
f 1 2 3 4
f 5 1 4 8
# start of the roof group
g roof
f 8 7 6 5
f 5 6 2 1
# start of the wall group
g wall
f 4 3 7 8
f 2 6 7 3

We use Wavefront OBJ as an example in this section, but keep in mind that most 3D mesh formats have the concept of an object ID within the file format.

Source filename

Another source of metadata is the source filename itself. It’s common to want to merge many small OBJ files into a single 3D Tiles Dataset. In that case, it’s good to know the source path of the original file. You can use it to distinguish between files, even if they use the same group IDs. You can also use it to distinguish files without group IDs.

A diagram showing OBJ files being processed into a single 3D Tiles dataset
Figure 3. The default metadata when several OBJ files are processed into a single 3D Tiles dataset.

Vertex object ID

The third metadata attribute added to the output 3D Tiles dataset is the so-called object ID of the vertex. This property is a guaranteed unique integer that you can use for selection by default.

To produce this property, the 3D Tiles Processing Engine takes all combinations of source paths and group IDs, and uses this information to generate a new unique ID. Those unique object IDs prevent situations where multiple input meshes share group IDs.

The default metadata model

You can find the default metadata model for the 3D Tiles Processing Engine in the TLcd3DTilesProcessorDataTypes class. The data types class contains these definitions by default:

  • TLcd3DTilesProcessorDataTypes.OBJECT_ID_PROPERTY: the uniquely generated ID that maps to an integer. You can use this property for selection.

  • TLcd3DTilesProcessorDataTypes.OBJECT_NAME_PROPERTY: the group ID of the vertex. It maps to a string.

  • TLcd3DTilesProcessorDataTypes.ORIGINAL_SOURCE_PATH_PROPERTY: the source path to the original mesh used for processing. It maps to a string.

The property OBJECT_NAME_PROPERTY refers to the group ID. The object name term is more generic, and applies to mesh formats in general. In this section, we referred to it as a group ID, because this is the specific term used in the OBJ formats. Other formats might use different terminology.

Customizing what metadata ends up in your dataset

The default data model applied to your output 3D Tiles dataset already covers many use cases. Sometimes though, you need to customize the default metadata output.

To customize the metadata created by default, set an ILcd3DTilesProcessorMetadataMapper on your TLcd3DTilesProcessorBuilder

Add a custom metadata mapper to your TLcd3DTilesProcessorBuilder
return TLcd3DTilesProcessorBuilder.newBuilder()
                                  .addInputFiles("path/to/input/folder/mesh.obj")
                                  .outputPath("path/to/output/folder/")
                                  //Adding a custom metadata provider to your mesh
                                  .metadataMapper(new Custom3DTilesMetadataMapper())
                                  .process()
                                  .get();

The idea behind a metadata mapper is that it functions as a hook. For each object ID in your original dataset, you get the chance to add metadata from any source.

To add metadata through the metadata mapper:

  1. Create a TLcdDataModel.

  2. Create a TLcdDataType within that model. Make sure this type has all the properties you need as output metadata.

  3. Implement ILcd3DTilesProcessorMetadataMapper.getDataObject() to create your data objects based on the default data object described in The default metadata model.

This figure displays a possible transformation that you can apply inside the ILcd3DTilesProcessorMetadataMapper.getDataObject() method. In the figure, we augment the default data object to contain a name, height, and description. We don’t preserve the object ID original source path.

A diagram showing an example transformation from default data object to custom data object
Figure 4. An example transformation of the default data object to a custom data object. This is the result of the code snippet with a custom metadata mapper

We get to that result from this code:

An example of a custom metadata mapper
/**
 * A custom metadata provider that adds a name, height and description to every object in the
 * original dataset.
 */
private static class Custom3DTilesMetadataMapper implements ILcd3DTilesProcessorMetadataMapper {

  private static final TLcdDataModel sCustomDataModel;
  private static final TLcdDataType sCustomDataType;

  private static final String NAME = "name";
  private static final String HEIGHT = "height";
  private static final String DESCRIPTION = "description";

  static {
    String typeName = "customType";
    TLcdDataModelBuilder dataModelBuilder = new TLcdDataModelBuilder("customDatamodel");
    TLcdDataTypeBuilder customType = dataModelBuilder.typeBuilder(typeName);

    // Add the generated unique ID from the default data type
    customType.addProperty(TLcd3DTilesProcessorDataTypes.OBJECT_ID_PROPERTY, TLcdCoreDataTypes.INTEGER_TYPE);
    customType.addProperty(NAME, TLcdCoreDataTypes.STRING_TYPE);
    customType.addProperty(HEIGHT, TLcdCoreDataTypes.FLOAT_TYPE);
    customType.addProperty(DESCRIPTION, TLcdCoreDataTypes.STRING_TYPE);

    sCustomDataModel = dataModelBuilder.createDataModel();
    sCustomDataType = sCustomDataModel.getDeclaredType(typeName);
    // Add primary key annotation to the unique ID
    sCustomDataType.addAnnotation(
        new TLcdPrimaryKeyAnnotation(sCustomDataType.getDeclaredProperty(TLcd3DTilesProcessorDataTypes.OBJECT_ID_PROPERTY))
    );
  }

  @Override
  public TLcdDataType getDataType(TLcdDataType aDefaultDataType) {
    return sCustomDataType;
  }

  @Override
  public ILcdDataObject getDataObject(ILcdDataObject aDefaultDataObject) {
    TLcdDataObject dataObject = new TLcdDataObject(sCustomDataType);
    // Copy the unique ID property over to the new data object.
    // This unique ID is useful to be able to select in the client.
    dataObject.setValue(TLcd3DTilesProcessorDataTypes.OBJECT_ID_PROPERTY, aDefaultDataObject.getValue(TLcd3DTilesProcessorDataTypes.OBJECT_ID_PROPERTY));
    String objectID = (String) aDefaultDataObject.getValue(TLcd3DTilesProcessorDataTypes.OBJECT_NAME_PROPERTY);
    String originalSourcePath = (String) aDefaultDataObject.getValue(TLcd3DTilesProcessorDataTypes.ORIGINAL_SOURCE_PATH_PROPERTY);

    // Use the original file path together with the object ID inside the
    // original file to retrieve the various properties of the dataset.
    // These methods can invoke queries to external files and databases if
    // needed to get the values.
    dataObject.setValue(NAME, getNameForObjectID(objectID, originalSourcePath));
    dataObject.setValue(HEIGHT, getHeightForObjectID(objectID, originalSourcePath));
    dataObject.setValue(DESCRIPTION, getDescriptionForObjectID(objectID, originalSourcePath));
    return dataObject;
  }

}

In the example code snippet, we assume that the metadata is read from an external source. This is common for 3D Tiles processing. The source data doesn’t always contain all the metadata you need, so you typically have to find it elsewhere, like in an external file or database.

You can use the ObjectID to identify which object you are currently augmenting with metadata to perform external lookup operations.

Binary versus JSON metadata

The OGC 3D Tiles format separates binary metadata from JSON metadata. It is important to understand the distinction between the two.

  • Binary metadata is any metadata of which the value has a fixed length that’s known upfront. Examples of binary metadata include boolean values, integers, floats, and doubles, but also floating point vectors like vec2, vec3, and vec4.

  • Everything else is JSON metadata. Examples of JSON metadata include strings, arrays, and complex object literals.

We recommend that you use binary metadata as much as possible in your models. Binary metadata has several important advantages:

  • It’s more compact to encode.

  • You can easily transfer it to the GPU on the client.

  • You can use it to create dynamic expressions in GPU code, for on-the-fly filtering, coloring, and selection.

You can optimize JSON metadata. These are some examples of optimizations:

  • Enums are typically encoded as JSON strings. To use them on a GPU expression, we recommend that you convert these to an integer instead.

  • GUIDs and other unique identifiers are often stored as strings. To use them for selection, we recommend that you convert them to integers instead. You can do that by mapping all possible identifiers to a Set while processing, and using the index of the GUID in the Set as the identifier within the dataset.

For more information on binary metadata versus JSON metadata, see the official OGC 3D Tiles specification

Special data types

When you are encoding OGC 3D Tiles metadata using ILcd3DTilesProcessorMetadataMapper, you can use several objects and primitives to model your TLcdDataModel. When you do so, the TLcd3DTilesProcessorBuilder automatically processes and converts your metadata to the most appropriate data type for OGC 3D Tiles.

As mentioned earlier, some data types such as String get converted to non-binary JSON metadata within OGC 3D Tiles.

However, you can also use some multi-component TLcdDataType objects inside the binary part of your OGC 3D Tiles dataset.

You can find those data types in the TLcdOGC3DTilesDataTypes class. They represent the VEC2, VEC3, and VEC4 components in OGC 3D Tiles batch tables:

  • TLcdOGC3DTilesDataTypes.VEC2_FLOAT_TYPE

  • TLcdOGC3DTilesDataTypes.VEC3_FLOAT_TYPE

  • TLcdOGC3DTilesDataTypes.VEC4_FLOAT_TYPE

  • TLcdOGC3DTilesDataTypes.VEC2_DOUBLE_TYPE

  • TLcdOGC3DTilesDataTypes.VEC3_DOUBLE_TYPE

  • TLcdOGC3DTilesDataTypes.VEC4_DOUBLE_TYPE

In Program: Add 3-component vectors to your binary data , we set up a custom ILcd3DTilesProcessorMetadataMapper to add a point to every single object in the dataset, using a float[] primitive.

Add a custom 3-component vector, such as a point or a normal, to your metadata
/**
 * A custom metadata provider that adds a 3-component floating point vector data to a dataset in binary space
 */
private static class Vec3MetadataMapper implements ILcd3DTilesProcessorMetadataMapper {

  private static final TLcdDataModel sCustomDataModel;
  private static final TLcdDataType sCustomDataType;

  private static final String POINT = "pointVec3";

  static {
    String typeName = "customType";
    TLcdDataModelBuilder dataModelBuilder = new TLcdDataModelBuilder("customDatamodel");
    TLcdDataTypeBuilder customType = dataModelBuilder.typeBuilder(typeName);

    // Add the generated unique ID from the default data type
    customType.addProperty(TLcd3DTilesProcessorDataTypes.OBJECT_ID_PROPERTY, TLcdCoreDataTypes.INTEGER_TYPE);
    customType.addProperty(POINT, TLcdOGC3DTilesDataTypes.VEC3_FLOAT_TYPE);

    sCustomDataModel = dataModelBuilder.createDataModel();
    sCustomDataType = sCustomDataModel.getDeclaredType(typeName);
    // Add primary key annotation to the unique ID
    sCustomDataType.addAnnotation(
        new TLcdPrimaryKeyAnnotation(sCustomDataType.getDeclaredProperty(TLcd3DTilesProcessorDataTypes.OBJECT_ID_PROPERTY))
    );
  }

  @Override
  public TLcdDataType getDataType(TLcdDataType aDefaultDataType) {
    return sCustomDataType;
  }

  @Override
  public ILcdDataObject getDataObject(ILcdDataObject aDefaultDataObject) {
    TLcdDataObject dataObject = new TLcdDataObject(sCustomDataType);
    // Copy the unique ID property over to the new data object.
    // This unique ID is useful to be able to select in the client.
    dataObject.setValue(TLcd3DTilesProcessorDataTypes.OBJECT_ID_PROPERTY, aDefaultDataObject.getValue(TLcd3DTilesProcessorDataTypes.OBJECT_ID_PROPERTY));

    //Adds a 3-component vector using a float array as the object. This will get encoded as binary metadata within OGC 3D Tiles
    dataObject.setValue(POINT, new float[]{1f,2f,3f});
    return dataObject;
  }

}

We strongly recommend that you use the datatypes in TLcdOGC3DTilesDataTypes starting with VEC to model any multi-dimensional vector data, such as positions, directions, and colors.

Because these data types end up in the binary part of OGC 3D Tiles metadata batch tables, clients can easily use them as filters in on-the-fly expressions.

Handling popular formats and tools

These articles discuss popular formats and tools used for producing 3D data. They also show you how to get that 3D data into processed OGC 3D Tiles output, together with some metadata.

Table 1. Mesh formats and tools instructions
Format or tool Link

IFC format

Data Formats: IFC

Autodesk 3D Max tool

Handling Autodesk 3D Max data

Autodesk Revit tool

Handling Autodesk Revit data

Blender tool

Handling Blender data

Many of these tools don’t support georeferences. This means that you must georeference your meshes yourself before you can put them on a map. See the guide on Processing meshes into OGC 3D tiles to find out how to manually add a georeference a dataset.

You can also process these datasets without a reference, but then you have to geolocate the datasets on the client. See the guide on Positioning non-referenced mesh and point cloud data to learn how to do that.