Package com.luciad.format.netcdf

Class TLcdNetCDFModelDecoder

java.lang.Object
com.luciad.format.netcdf.TLcdNetCDFModelDecoder
All Implemented Interfaces:
ILcdInputStreamFactoryCapable, ILcdDataSourceModelDecoder, ILcdModelDecoder, ILcdStatusSource
@LcdService(service=ILcdModelDecoder.class, priority=20000) public class TLcdNetCDFModelDecoder extends Object implements ILcdModelDecoder, ILcdDataSourceModelDecoder, ILcdStatusSource, ILcdInputStreamFactoryCapable
Decodes NetCDF, NcML, GRIB and HDF files.

A NetCDF file includes one or more NetCDF variables, each of which represents a set of scientific measurements sampled on specific spatial coordinates, with or without times and/or vertical positions. A NetCDF variable can for example contain temperature values of oceans for different locations, times and altitudes. A NetCDF model for this variable contains an image with temperature values that can vary over different times and altitudes.

An NcML file is an XML representation of NetCDF metadata and points to a NetCDF file. Hence, the XML file always needs to be accompanied with the actual NetCDF file.

This model decoder supports NetCDF 3 and NetCDF 4 (classic and HDF5) files that follow;

  • NetCDF CF (Climate & Forecast) conventions.
  • WRF (The Weather Research & Forecasting Model) NetCDF.
In addition, the data has to be regularly gridded.

Input files

FileRequiredEntry pointDescription
*.nc, *.netcdfXXThe NetCDF file
*.xmlXNcML File
*.grb, *.grib, *.dat, *.grb2, *.grib2XGRIB File
*.hdf, *.h5, *.he5, *.hdf5XXHDF5 File

Supported file transfer protocols

  • This model decoder decodes data from a file system.

Caching

This model decoder will generate certain cache files when decoding a source (e.g. GRIB index files). If the default directory for these cache files needs to be changed, it can be done by setting the system property com.luciad.netcdf.cachedir. Setting this property prevents issues with creating cache files in case the default cache directory is non-writable. The system property can be set by providing it as a VM argument, e.g. -Dcom.luciad.netcdf.cachedir=<a-valid-path>. The directory provided in the system property will be created if it is not available.

Model structure

  • This model decoder creates a model for every NetCDF coverage variable.
  • If the input NetCDF file has more than one coverage, the resulting model is an ILcdModelTreeNode.
  • A multi variable NetCDF file can also be queried for specific coverages by using a file:// URI with query parameters.
  • All models returned by this model decoder implement ILcd2DBoundsIndexedModel.

Model descriptor

Model reference

Model elements

See TLcdNetCDFFilteredModel for more information.

Sample code

Example: Decoding a source
    String source = "Data/NetCDF/cloud_cover.nc";

    // decode the model
    TLcdNetCDFModelDecoder decoder = new TLcdNetCDFModelDecoder();
    ILcdModel model = decoder.decode(source);
Example: Decode a specific set of coverages
You can either use a string which includes query parameters 
    TLcdNetCDFModelDecoder decoder = new TLcdNetCDFModelDecoder();
    String source = "Data/NetCDF/wind_u.nc";
    //Use a query parameter to indicate that only the U-component should be decoded
    ILcdModel model = decoder.decode(source + "?variable=u-component_of_wind_isobaric");
or specify a set of data sources 
    TLcdNetCDFModelDecoder decoder = new TLcdNetCDFModelDecoder();
    String source = "Data/NetCDF/wind_u.nc";

    List<ILcdDataSource> allDataSources = decoder.discoverDataSources(source);
    //Filter the available data sources
    List<TLcdNetCDFDataSource> filteredDataSources =
        allDataSources.stream()
                      .filter(dataSource -> dataSource instanceof TLcdNetCDFDataSource)
                      .map(dataSource -> (TLcdNetCDFDataSource) dataSource)
                      .filter(dataSource -> dataSource.getName().toLowerCase().contains("u-component"))
                      .collect(Collectors.toList());
    //Only pass the data sources in which we are interested to the model decoder
    ILcdModel model = decoder.decodeSource(new TLcdNetCDFMultiBandDataSource(filteredDataSources));
Example: Retrieve measurements
    //Decode the model
    List<TLcdISO19103Measure> measurementsToReturn = new ArrayList<>();
    String source = "Data/NetCDF/cloud_cover.nc";
    TLcdNetCDFModelDecoder decoder = new TLcdNetCDFModelDecoder();
    ILcd2DBoundsIndexedModel model = (ILcd2DBoundsIndexedModel) decoder.decode(source);

    //Create an ALcdMeasureProvider by using the ILcdModelMeasureProviderFactory instances registered as service
    Iterator<ILcdModelMeasureProviderFactory> iterator = TLcdServiceLoader.getInstance(ILcdModelMeasureProviderFactory.class).iterator();
    ALcdMeasureProvider measureProvider = null;
    while (iterator.hasNext() && measureProvider == null) {
      ILcdModelMeasureProviderFactory factory = iterator.next();
      measureProvider = factory.createMeasureProvider(model);
    }

    //Query the ALcdMeasureProvider for the measurements at a certain location
    if (measureProvider != null) {
      ILcdModelReference modelReference = model.getModelReference();
      ILcdPoint center = model.getBounds().getCenter();

      ALcdMeasureProvider.Parameters params = ALcdMeasureProvider.Parameters.newBuilder().build();
      TLcdISO19103Measure[] measurements = measureProvider.retrieveMeasuresAt(center, modelReference, params);
      for (TLcdISO19103Measure measurement : measurements) {
        //do something with the measurement
        System.out.println("measurement = " + measurement);
      }
    }
Example: filter a NetCDF model
    String source = "Data/NetCDF/cloud_cover.nc";
    TLcdNetCDFModelDecoder decoder = new TLcdNetCDFModelDecoder();
    ILcdModel model = decoder.decode(source);

    if (model instanceof ILcdMultiDimensionalModel) {
      // get the time axis if you don't have it already
      TLcdDimensionAxis<Date> timeAxis =
          ((ILcdMultiDimensional) model).getDimensions()
                                        .stream()
                                        .map(ILcdDimension::getAxis)
                                        .filter(axis -> axis.getType().equals(Date.class))
                                        .map(axis -> (TLcdDimensionAxis<Date>) axis)
                                        .findFirst()
                                        .orElse(null);

      Date startDate = ((ILcdMultiDimensionalModel) model).getDimensions()
                                                          .stream()
                                                          .filter(dimension -> dimension.getAxis() == timeAxis)
                                                          .map(ILcdDimension::getUnionOfValues)
                                                          .map(interval -> (TLcdDimensionInterval<Date>) interval)
                                                          .map(TLcdDimensionInterval::getMin)
                                                          .findFirst()
                                                          .orElse(null);

      // create a filter for a specific date
      TLcdDimensionFilter filter = TLcdDimensionFilter
          .newBuilder()
          .filterDimension(timeAxis, TLcdDimensionInterval.createSingleValue(Date.class, startDate))
          .build();

      // apply the filter within a write lock
      try (TLcdLockUtil.Lock lock = TLcdLockUtil.writeLock(model)) {
        ((ILcdMultiDimensionalModel) model).applyDimensionFilter(filter, ILcdModel.FIRE_LATER);
      } finally {
        // fire model changes outside lock
        model.fireCollectedModelChanges();
      }
    }
For more information on how to integrate and manipulate NetCDF data into your application, refer to the NetCDF developer's guide.

Thread safety

  • The decoded models are thread-safe for read access when taking a read lock.
Since:
2015.0

  • Constructor Details

    • TLcdNetCDFModelDecoder

      public TLcdNetCDFModelDecoder()
      Create a new decoder instance.

  • Method Details

    • getMultiBandImageCreation

      public TLcdNetCDFModelDecoder.MultiBandImageCreation getMultiBandImageCreation()
      Returns when multi-band images are created. Multi-band images combine related models, such as the x and y components of a wind measurement. They are typically visualized with a grid of icons.
      Returns:
      when multi-band images are created, AUTO or NEVER.
      See Also:

    • setMultiBandImageCreation

      public void setMultiBandImageCreation(TLcdNetCDFModelDecoder.MultiBandImageCreation aMultiValuedRasterCreation)

      Configures the creation of multi-band images. Multi-band images combine related datasets, such as the x and y components of a wind measurement. They are typically visualized with a grid of icons.

      When loading GRIB 1 files, setting this flag to AUTO will also make sure that the decode method looks for other related GRIB files next to the one being decoded. For example, when passing a GRIB file that only contains 'wind U' data, it will also look for a file that contains 'Wind V' data. This automatic combining is based on a heuristic that assumes that the GRIB parameter ID is part of the file name.

      In case setting this flag to AUTO fails to automatically create multi-band images, it is possible to combine them manually. This can be done by passing a TLcdNetCDFMultiBandDataSource to the decodeSource method.

      Parameters:
      aMultiValuedRasterCreation - used to configure when multi-band images are created, AUTO or NEVER
      See Also:

    • setCalculateMinMaxValue

      public void setCalculateMinMaxValue(boolean aCalculateMinMaxValue)

      Configures whether this decoder calculates a min and max value for the ALcdBandSemantics that is set on the model element.

      By default, this is true.

      Note however that the min and max values are only calculated on a small part of the model (to avoid reading the entire model in memory). This may lead to values that are incorrect. When using functionality that depends on the correctness of these values (for example image processing), it may be more desirable to not calculate these values at all.

      Parameters:
      aCalculateMinMaxValue - whether to calculate a min and max value.
      Since:
      2025.0.05

    • isCalculateMinMaxValue

      public boolean isCalculateMinMaxValue()

      Returns whether this decoder calculates a min and max value for the ALcdBandSemantics that is set on the model element.

      Returns:
      whether this decoder calculates a min and max value for the ALcdBandSemantics that is set on the model element.
      Since:
      2025.0.05
      See Also:

    • getDisplayName

      public String getDisplayName()
      Description copied from interface: ILcdModelDecoder
      Returns a short, displayable name for the format that is decoded by this ILcdModelDecoder.
      Specified by:
      getDisplayName in interface ILcdModelDecoder
      Returns:
      the displayable name of this ILcdModelDecoder.

    • canDecodeSource

      public boolean canDecodeSource(String aSource)
      Description copied from interface: ILcdModelDecoder
      Checks whether this model decoder can decode the specified data source. It is acceptable for this method to return true for a source name while decode throws an exception for that same source name.

      For performance reasons, we strongly recommend that this will only be a simple test. For example: check the file extension of a file, but not that the file exists or contains expected content.

      Specified by:
      canDecodeSource in interface ILcdModelDecoder
      Parameters:
      aSource - the data source to be verified; typically a file name or a URL.
      Returns:
      true if this decoder can likely decode the data specified by the source name, false otherwise.
      See Also:

    • discoverDataSources

      public List<ILcdDataSource> discoverDataSources(String aSource) throws IOException

      Fetch the data sources from the given NetCDF file path.

      The data sources are either TLcdNetCDFDataSource or TLcdNetCDFMultiBandDataSource instances. Data sources are combined in a TLcdNetCDFMultiBandDataSource if they are related. To only get single-band sources, set the multi-band image creation of this decoder to TLcdNetCDFModelDecoder.MultiBandImageCreation.NEVER using the setMultiBandImageCreation(com.luciad.format.netcdf.TLcdNetCDFModelDecoder.MultiBandImageCreation) method

      To identify the type of data that is represented by a TLcdNetCDFDataSource instance, you can retrieve the full name of the NetCDF variable using the TLcdNetCDFDataSource.getName() method.

      To identify the type of data that is represented by a TLcdNetCDFMultiBandDataSource instance, you can retrieve data sources that are part of the multi-band source by using the TLcdNetCDFMultiBandDataSource.getDataSources() method.

      Specified by:
      discoverDataSources in interface ILcdModelDecoder
      Parameters:
      aSource - a path to a NetCDF file.
      Returns:
      A list of data sources available in the given source file. May be empty but not null. Note that not all data sources returned may be supported by this decoder.
      Throws:
      IOException - if the decoder can't decode the source file.

    • canDecodeSource

      public boolean canDecodeSource(ILcdDataSource aDataSource)

      Checks whether this model decoder can decode the data source(s), identified by the passed ILcdDataSource.

      For performance reasons, we strongly recommend that this will only be a simple test. For example: check the instance class of aDataSource, or check the file extension if it is a TLcdDataSource.

      The default implementation of this method will check if the given ILcdDataSource is a TLcdDataSource. If not, this method returns false. Otherwise, it delegates the source to the ILcdModelDecoder.canDecodeSource(String) method.

      Supported data sources are:

      Specified by:
      canDecodeSource in interface ILcdModelDecoder
      Parameters:
      aDataSource - the ILcdModelSource to be verified.
      Returns:
      true if this decoder can likely decode the data specified by aDataSource, false otherwise.
      See Also:

    • decode

      public ILcdModel decode(String aSource) throws IOException
      Description copied from interface: ILcdModelDecoder
      Creates a new model from the given data source.
      Specified by:
      decode in interface ILcdModelDecoder
      Parameters:
      aSource - the data source to be decoded; typically a file name or a URL.
      Returns:
      A model containing the decoded data. While null is allowed, implementors are advised to throw an error instead.
      Throws:
      IOException - for any exceptions caused by IO problems or invalid data. Since decoding invalid data almost always results in RunTimeExceptions (NullPointerException, IndexOutOfBoundsException, IllegalArgumentException, ...) on unexpected places, implementations are advised to catch RuntimeExceptions in their decode() method, and wrap them into an IOException, as illustrated in the code snippet below. 
      
   public ILcdModel decode( String aSourceName ) throws IOException {
      try (InputStream input = fInputStreamFactory.createInputStream(aSourceName)) {
         // Perform decoding ...
      } catch (RuntimeException e) {
         throw new IOException(e);
      }
   }
      See Also:

    • decodeModelMetadata

      public TLcdModelMetadata decodeModelMetadata(ILcdDataSource aDataSource) throws IOException
      Decodes the model metadata of a NetCDF source.

      Supported data sources are:

      For multi-band sources, one of the sources is used as the entry point source of the model metadata. All other sources will be set as the supporting sources of the model metadata.

      Specified by:
      decodeModelMetadata in interface ILcdModelDecoder
      Parameters:
      aDataSource - the data source to be decoded.
      Returns:
      a TLcdModelMetadata instance containing the model metadata of the NetCDF model of the source.
      Throws:
      IOException - If the given source cannot be decoded for some reason.
      See Also:

    • decodeSource

      public ILcdModel decodeSource(ILcdDataSource aDataSource) throws IOException

      Creates a new model from the given data source.

      By default, this method:

      Supported data sources are:

      Specified by:
      decodeSource in interface ILcdModelDecoder
      Parameters:
      aDataSource - the ILcdDataSource to be decoded.
      Returns:
      a model containing the decoded data. While null is allowed, implementors are advised to throw an error instead.
      Throws:
      IOException - for any exceptions caused by IO problems or invalid data. Since decoding invalid data almost always results in RunTimeExceptions (NullPointerException, IndexOutOfBoundsException, IllegalArgumentException, ...) on unexpected places, implementations are advised to catch RuntimeExceptions in their decode() method, and wrap them into an IOException, as illustrated in the code snippet below. 
      
 public ILcdModel decodeSource(ILcdDataSource aDataSource) throws IOException {
   try {
     // Perform decoding ...
   } catch (RuntimeException e) {
     throw new IOException(e);
   }
 }
      See Also:

    • addStatusListener

      public void addStatusListener(ILcdStatusListener aListener)
      Description copied from interface: ILcdStatusSource

      Registers the given listener so it will receive status events from this source.

      In case you need to register a listener which keeps a reference to an object with a shorter life-time than this status source, you can use a ALcdWeakStatusListener instance as status listener.

      Specified by:
      addStatusListener in interface ILcdStatusSource
      Parameters:
      aListener - The listener to be notified when the status has changed.

    • removeStatusListener

      public void removeStatusListener(ILcdStatusListener aListener)
      Description copied from interface: ILcdStatusSource
      Removes the specified listener so it is no longer notified.
      Specified by:
      removeStatusListener in interface ILcdStatusSource
      Parameters:
      aListener - The listener to remove.

    • setInputStreamFactory

      public void setInputStreamFactory(ILcdInputStreamFactory aInputStreamFactory)
      Sets the input stream factory to be used. When the input stream factory is set,
      1. This model decoder reads all the data from the created stream and stores it in a temporary file. The temporary file is created by File.createTempFile(String, String) in the OS temporary files directory.
      2. The temporary file is used as the NetCDF source file.
      3. The temporary file is deleted either when the root NetCDF model is disposed or JVM terminates normally.
      Note: Ensure that the running process has enough privileges to read and write temporary files directory to use an input stream as a NetCDF data source.
      Specified by:
      setInputStreamFactory in interface ILcdInputStreamFactoryCapable
      Parameters:
      aInputStreamFactory - the input stream factory to be used.
      See Also:

    • getInputStreamFactory

      public ILcdInputStreamFactory getInputStreamFactory()
      Description copied from interface: ILcdInputStreamFactoryCapable
      Returns the input stream factory that is used.
      Specified by:
      getInputStreamFactory in interface ILcdInputStreamFactoryCapable
      Returns:
      the input stream factory that is used.