How WFS data filtering works

In line with the OGC WFS standard, the WFS server supports the filtering of data through OGC Filters. If clients include an OGC Filter in their requests, the WFS server automatically applies it to the requested data. It forwards the filter to the ILcdModel implementation behind the data, through its query method. That way, you can filter the data directly at the source, because the model implementation holds the information for accessing it. For instance, the model for a database format automatically converts the filter to a SQL query, which filters the data directly in the database.

A TLcdWFSFilteredModelFactory sets up this data filtering pipeline.

When and how to customize WFS data filtering

These sections discuss use cases for which you need to customize the WFS data filtering pipeline. They also show you how you can do that for each case.

Extending OGC Filter with a custom function

The OGC Filter standard offers an extension mechanism. You can use it to plug in functions that support custom filtering logic. To learn how to create such a function, and plug it into the main OGC Filter evaluator class TLcdOGCFilterEvaluator, see How to add a custom function to OGC Filter. The WFS server automatically picks up such functions. Program: Adding an OGC Filter function illustrates this use case by creating a function to calculate the area of a feature.

Program: Adding an OGC Filter function
static {
  // Register a custom OGC Filter function to calculate the geodesic area of a feature in square meters.
  // Supported geometries are polygons, complex polygons and shape lists containing them.
  TLcdEllipsoid ellipsoid = new TLcdEllipsoid();
  TLcdOGCFilterEvaluator.registerDefaultFunction(TLcdXMLName.getInstance( new QName("geodesicArea")), new ILcdEvaluatorFunction() {

    @Override
    public int getArgumentCount() {
      // No client arguments needed for this function.
      // We just calculate the geodesic area based on the features residing in the WFS.
      return 0;
    }

    @Override
    public Object apply(Object[] aArguments, Object aCurrentObject, TLcdOGCFilterContext aOGCFilterContext) {
      // We assume that the supported objects implement ILcdShape
      if (aCurrentObject instanceof ILcdShape) {
        return calculateArea((ILcdShape) aCurrentObject);
      }
      // In all other cases, we return 0.
      else {
        return 0;
      }

    }

    private double calculateArea(ILcdShape aShape) {
      try {
        return TLcdEllipsoidUtil.geodesicArea(aShape, ellipsoid);
      } catch (IllegalArgumentException e) {
        return 0;
      }
    }

  });
}

Customizing the OGC Filter evaluation

To evaluate an OGC Filter, you need to interpret the content of a filter. The filter consists of a set of feature identifiers or conditions that operate on the feature properties. You can customize both aspects to meet your filter evaluation needs.

Customizing the evaluation of the feature identifier

To customize the evaluation of feature identifiers, you can create your own ILcdOGCFeatureIDRetriever. You use that interface to find the unique identifier (ID) of each feature in your data set. The default implementation relies on the optional TLcdPrimaryKeyAnnotation to find the property that has the unique identifier. To change that for your data, create your own feature ID retriever, and return it in the getFeatureIDRetriever method of ILcdWFSFeatureType when you define the features types of your WFS server.

Customizing the evaluation of feature conditions

To customize the evaluation of feature conditions, you create an implementation of ILcdOGCFilterEvaluator. To install it on the WFS server, create your own ILcdWFSFilteredModelFactory that returns the evaluator in its factory method createFilterEvaluator. You can then register the filtered model factory with the WFS server through the factory method createFilteredModelFactory in ALcdOGCWFSCommandDispatcherFactory.

Customizing the OGC Filter propagation to the model

By default, the WFS server forwards a filter to the model of the requested feature type. It’s then up to the model implementation to evaluate the filter, and apply it to the data source. If you customized the OGC Filter evaluation logic, the WFS server does the evaluation itself, because the data source of the model, a database for example, doesn’t know about the customization.

You can customize that behavior by creating your own implementation of the method createFilteredModel in ILcdWFSFilteredModelFactory. To use it in the WFS server, return it in the factory method createFilteredModelFactory in ALcdOGCWFSCommandDispatcherFactory. As a starting point, Program: Creating a custom filtered model shows the creation of a fully functional filtered model that forwards the filter to the model.

Program: Creating a custom filtered model
  private ILcdOGCFilterEvaluator fFilterEvaluator = new TLcdOGCFilterEvaluator();

  @Override
  public TLcdOGCFilterCapabilities getFilterCapabilities() {
    return fFilterEvaluator.getFilterCapabilities();
  }

  @Override
  public ILcdModel createFilteredModel(ILcdModel aSourceModel, TLcdWFSGetFeatureConstraints aConstraints) throws TLcdWFSServiceException {
    if (aConstraints.getPropertyCount() > 0) {
      throw new IllegalStateException("Property name filtering not supported by this ILcdWFSFilteredModelFactory");
    }

    return createFilteredModel(
        aSourceModel,
        aConstraints.getFeatureType(),
        aConstraints.getFilter(),
        aConstraints.getSortBy(),
        aConstraints.getMaxFeatures(),
        aConstraints.getRequestContext()
    );
  }

  private ILcdModel createFilteredModel(final ILcdModel aSourceModel, final ILcdWFSFeatureType aFeatureType,
                                        TLcdOGCFilter aFilterDefinition, TLcdOGCSortBy aSortBy,
                                        int aMaxFeatures, TLcdWFSRequestContext aRequestContext) throws TLcdWFSServiceException {

    ILcdModel filteredModel = aSourceModel;

    if ((aFilterDefinition != null) || (aMaxFeatures >= 0) || (aSortBy != null)) {
      filteredModel = new FilteredModel(aSourceModel, null, aFilterDefinition != null ? aFilterDefinition.getCondition() : null, null, aSortBy, aMaxFeatures);
    }
    return filteredModel;
  }

  private static class FilteredModel implements ILcdIntegerIndexedModel, ILcdBounded {

    private final ILcdModel fModel;
    private final ILcdOGCCondition fOGCFilter;
    private final ILcdBounds fBounds;
    private final ILcdFilter fFilter;
    private final TLcdOGCSortBy fSortBy;
    private final int fLimit;

    private ILcdBounds fCachedBounds;
    private SoftReference<List<Object>> fCachedElementsReference = new SoftReference<>(null);

    /**
     * Creates a new filtered model.
     *
     * @param aModel The original model.
     * @param aBounds The output bounds.  Note that this is only for getBounds.  Add a BBOX operator to the OGC filter below.  Optional: if null, the bounds will be calculated from the data.
     * @param aOGCFilter An OGC filter, including BBOX if applicable.  Use null if not applicable.
     * @param aFilter A non-OGC filter.  Use null if not applicable.
     * @param aSortBy A sorting order.  Use null if not applicable.
     * @param aMaxFeatures Max features.  Use -1 if not applicable.
     */
    public FilteredModel(ILcdModel aModel, ILcdBounds aBounds, ILcdOGCCondition aOGCFilter, ILcdFilter aFilter, TLcdOGCSortBy aSortBy, int aMaxFeatures) {
      fModel = aModel;
      fOGCFilter = aOGCFilter;
      fBounds = aBounds;
      fFilter = aFilter;
      fSortBy = aSortBy;
      fLimit = aMaxFeatures;
    }

    public ILcdModel getModel() {
      return fModel;
    }

    // Implementations for ILcdModel

    @Override
    public ILcdModelDescriptor getModelDescriptor() {
      return fModel.getModelDescriptor();
    }

    @Override
    public ILcdModelEncoder getModelEncoder() {
      return fModel.getModelEncoder();
    }

    @Override
    public ILcdModelReference getModelReference() {
      return fModel.getModelReference();
    }

    @Override
    public boolean canAddElement(Object aObject) {
      return fModel.canAddElement(aObject);
    }

    @Override
    public void addElement(Object aObject, int aEventMode) {
      fModel.addElement(aObject, aEventMode);
      invalidate();
    }

    @Override
    public void addElements(Vector aObjects, int aEventMode) {
      fModel.addElements(aObjects, aEventMode);
      invalidate();
    }

    @Override
    public boolean canRemoveElement(Object aObject) {
      return fModel.canRemoveElement(aObject);
    }

    @Override
    public void removeElement(Object aObject, int aEventMode) {
      fModel.removeElement(aObject, aEventMode);
      invalidate();
    }

    @Override
    public void removeElements(Vector aObjects, int aEventMode) {
      fModel.removeElement(aObjects, aEventMode);
      invalidate();
    }

    @Override
    public void removeAllElements(int aEventMode) {
      fModel.removeAllElements(aEventMode);
      invalidate();
    }

    @Override
    public void elementChanged(Object aObject, int aEventMode) {
      fModel.elementChanged(aObject, aEventMode);
    }

    @Override
    public void elementsChanged(Vector aObjects, int aEventMode) {
      fModel.elementsChanged(aObjects, aEventMode);
    }

    @Override
    public void fireCollectedModelChanges() {
      fModel.fireCollectedModelChanges();
    }

    @Override
    public void addModelListener(ILcdModelListener aModelListener) {
      fModel.addModelListener(aModelListener);
    }

    @Override
    public void removeModelListener(ILcdModelListener aModelListener) {
      fModel.removeModelListener(aModelListener);
    }

    @Override
    public void dispose() {
      fModel.dispose();
    }

    // Implementations for ILcdBounded

    @Override
    public ILcdBounds getBounds() {
      if (fBounds != null) {
        return fBounds;
      }
      if (fCachedBounds == null) {
        fCachedBounds = calculateBounds();
      }
      return fCachedBounds;
    }

    // Implementations for ILcdModel

    @Override
    public Enumeration elements() {
//      Thread.dumpStack();
      return enumerate(getSortedAndFilteredElementsCached());
    }

    // Implementations for ILcdIntegerIndexedModel

    @Override
    public int size() {
      return getSortedAndFilteredElementsCached().size();
    }

    @Override
    public Object elementAt(int aIndex) {
      return getSortedAndFilteredElementsCached().get(aIndex);
    }

    @Override
    public int indexOf(Object aObject) {
      return getSortedAndFilteredElementsCached().indexOf(aObject);
    }

    // Actual work

    private List<Object> getSortedAndFilteredElementsCached() {
      List<Object> elements = fCachedElementsReference.get();
      if (elements == null) {
        long start = System.currentTimeMillis();
        try (Stream<Object> stream = doQuery()) {
          elements = stream.collect(Collectors.toList());
        }
        fCachedElementsReference = new SoftReference<>(elements);
      }
      return elements;
    }

    private Stream<Object> doQuery() {
      Thread.dumpStack();

      ILcdModel.Query query = filter(fOGCFilter).sorted(fSortBy);

      if (fLimit >= 0 && fFilter == null) { // only apply back-end limit if no post-filtering is done
        query = query.limit(fLimit);
      }

      Stream<Object> results = fModel.query(query);

      if (fFilter != null) {
        results = results.filter(fFilter);
      }
      if (fLimit >= 0) {
        results = results.limit(fLimit);
      }

      return results;
    }

    private void invalidate() {
      fCachedBounds = null;
      fCachedElementsReference = new SoftReference<>(null);
    }

    private ILcdBounds calculateBounds() {
      ILcd2DEditableBounds modelBounds = null;
      for (Object element : getSortedAndFilteredElementsCached()) {
        ILcdBounds elementBounds = ALcdBounds.fromDomainObject(element);
        if (elementBounds != null) {
          if (modelBounds == null) {
            modelBounds = elementBounds.cloneAs2DEditableBounds();
          } else {
            modelBounds.setTo2DUnion(elementBounds);
          }
        }
      }
      return modelBounds;
    }
  }

  private static <T> Enumeration<T> enumerate(Iterator<T> s) {
    return new Enumeration<T>() {
      @Override
      public boolean hasMoreElements() {
        return s.hasNext();
      }

      @Override
      public T nextElement() {
        return s.next();
      }
    };
  }

  private static <T> Enumeration<T> enumerate(Iterable<T> s) {
    return enumerate(s.iterator());
  }