Asynchronous painting is an important high-level application of concurrency. Painting data in a view can be slow for many reasons. Sometimes, simply retrieving the data takes a considerable amount of time because I/O is slow, when the data is retrieved across a network or from large databases, for example. In some cases, the data still has to be decoded on the fly, imposing additional overhead. And for some data types, the painting process itself is complex and relatively time-consuming. For example, painting large rasters that are warped on the fly is a non-trivial operation.
In simple applications, repainting components of the GUI is done synchronously, in a single thread. Since the view is also a GUI component, the entire user interface freezes until the view has been painted when it is updated. The experience of the end-user can be greatly enhanced if the user interface remains responsive during each repaint. This can be achieved by painting (parts of) the view in one or more background image buffers, in one or more separate threads. When the threads have finished painting, they can copy the buffers to the actual view. Even though the view may lag behind, the user can continue to interact with the rest of the user interface.
Views need to be painted in the event dispatch thread (AWT thread, Swing thread) to make use of asynchronous painting.
Creating an asynchronous painting wrapper
LuciadLightspeed provides support for asynchronous painting at the layer level. Selected layers or groups of layers can be painted in buffer images in different threads. If some buffer images are not up to date yet, a best effort is made to provide previews of their contents. The view is updated automatically whenever the final buffer images are ready.
The support for asynchronous painting is based on a wrapper for a standard
ILcdGXYLayer. Whenever a layer is created for which the painting could be slow, the code can create an asynchronous wrapper, as shown
in Program: Creating an asynchronous layer for a slow layer. The wrapper can then be added to the view like any other layer.
TLcdGXYAsynchronousLayerWrapper asynchronousGeoTIFFLayer = new TLcdGXYAsynchronousLayerWrapper(geoTIFFLayer, null);
Once you wrap a layer for asynchronous painting, the asynchronous painting thread can access the original layer anytime. It is thus unsafe to access the original layer (including its painters and label painters) from any other thread, unless you take specific precautions. The layer wrapper itself can be accessed from the event dispatch thread. Accessing asynchronously painted layers provides more information on how to access your original layer after it has been wrapped. See Threading and locking in LuciadLightspeed for a more general overview of the concurrency best practices in LuciadLightspeed.
LuciadLightspeed offers the following asynchronous layer wrapper implementations:
TLcdGXYAsynchronousEditableLabelsLayerWrapperis the best implementation to use with layers implementing
ILcdGXYEditableLabelsLayer, such as
TLcdGXYLayer. The wrapper ensures that all LuciadLightspeed functionality can transparently access the advanced labeling functionality offered by the wrapped layer.
TLcdGXYAsynchronousLayerTreeNodeWrapperis best used with layers implementing
ILcdLayerTreeNode. The wrapper ensures that all LuciadLightspeed functionality can transparently access the tree functionality offered by the wrapped layer. Use
TLcdGXYAsynchronousEditableLabelsLayerTreeNodeWrapperif the layer tree node also implements
ILcdGXYEditableLabelsLayer, such as
TLcdGXYLayerTreeNode. Note that the wrapper only makes the layer itself paint asynchronously, not its children. If you want the children to be painted asynchronously, you can wrap them individually.
TLcdGXYAsynchronousLayerWrappercan be used for layers that neither implement
You can use the factory method
Accessing asynchronously painted layers
ILcdGXYAsynchronousLayerWrapper provides access to the standard properties of
ILcdGXYLayer, which can be safely read and written from the Event Dispatch Thread (EDT). It also provides standard methods like
applyOnInteract. In the default implementation, these methods delegate to the wrapped layer. To ensure thread-safety, they will block if
the layer is being painted in the painting thread.
The layer interface
ILcdGXYLayer also provides access to the painter and label painter for any given object. The default implementation of the asynchronous
layer wrapper returns a clone of the painter returned by the inner layer. It blocks if the layer is being painted in the painting
thread, in order to safely create the clone. The clone can then be used safely, assuming the painter class provides a sufficiently
deep clone. If you use your own painters, you can use
ILcdDeepCloneable to handle recursive cloning loops.
If an application wants to access the wrapped layer directly, the simplest way is to retrieve it from the layer wrapper, with
getGXYLayer. It can then, for example, read the layer’s properties. It is, however, not safe to modify the layer properties this way.
Or more generally, it is not safe to invoke methods that are not thread-safe, since the layer may be painted in the painting
thread of a paint queue.
A safe way to retrieve and to modify non-standard layer properties is to ask the layer wrapper to execute the modification
code at a safe point in time, on an appropriate thread. The interface
ILcdGXYAsynchronousLayerWrapper provides the methods
invokeNowOnGXYLayer for this purpose:
invokeAndWaitOnGXYLayerexecutes a given runnable on the caller’s thread when the paint queue is not painting. The method receives the wrapped layer as an argument and it can safely work on it (see Figure 1, “The method
invokeAndWaitOnGXYLayerexecutes a runnable on the caller’s thread, in between painting operations of the paint queue on the painting thread”).
invokeLaterOnGXYLayerexecutes a given runnable on the asynchronous painting thread. The method receives the wrapped layer as an argument. It can safely work on this layer, since the painting thread is obviously not painting at this point (see Figure 2, “The method
invokeLaterOnGXYLayerexecutes a runnable on the painting thread, in between painting operations of the paint queue”). The method must not invoke any methods on the event dispatch thread with
SwingUtilities.invokeAndWait, since this might result in dead-locks. Instead, use the method
invokeLaterOnGXYLayerInEDTto safely execute a given runnable on the event dispatch thread.
invokeNowOnGXYLayerexecutes a given runnable on the caller’s thread, much like
invokeAndWaitOnGXYLayer. However, instead of waiting for any paint operations to finish, it can invoke the method while painting operations are in progress. The runnable can therefore only safely invoke methods that do not interfere with the
paintmethod(see Figure 3, “The method
invokeNowOnGXYLayerexecutes a runnable on the caller’s thread, possibly overlapping with painting operations of the paint queue”).
invokeAndWaitOnGXYLayerexecutes a runnable on the caller’s thread, in between painting operations of the paint queue on the painting thread
invokeLaterOnGXYLayerexecutes a runnable on the painting thread, in between painting operations of the paint queue
invokeNowOnGXYLayerexecutes a runnable on the caller’s thread, possibly overlapping with painting operations of the paint queue
Providing transparent access to a custom layer interface explains another, more advanced alternative to accessing the wrapped layer’s functionality: extending the layer wrapper.
Editing the model of a wrapped layer
When editing the model of a wrapped layer you need to do this in a thread-safe way since the model is being painted in a separate painting thread. The layer wrapper and the paint queue guard any read access by means of read locks on the model, as described in Accessing asynchronously painted layers. Any code that modifies the model or its elements therefore has to make sure that it obtains a write lock on the model, as shown in Threading and locking in LuciadLightspeed.
Memory and performance considerations
Since each image buffer has the same size as the view, the required amount of memory can be considerable. It can therefore be useful to share paint queues between asynchronous layer wrappers. The contents of the actual layers are then painted in the same threads and in the same image buffers. Shared paint queues also reduce view repaints for the compositing created images, which can be important when the view contains slow, synchronously painted layers.
On the downside, sharing a paint queue implies that view repaints are as slow as the combined repaints of the layers. Layers that share a paint queue must also be consecutive. Otherwise their image buffers cannot be composited properly in the view. Customizing paint queue assignments explains how to simplify the non-trivial task of assigning paint threads and buffers to asynchronously painted layers by using a paint queue manager.
A paint queue also has a flag to determine whether it should create image buffers for all painting modes or not. It is possible
to reduce the number of buffers in order to reduce the memory usage. Depending on the type of layer,
this comes at the cost of some performance or quality. By default, the value
EVERYTHING is specified. The paint queue then creates image buffers for all painting modes. The value
BODIES_AND_SKIP instructs the paint queue to only create an image buffer for painting the bodies of the model elements. The labels and selection
are painted synchronously, but only when the image buffer is ready. To avoid the continuous appearing and disappearing of
the labels and selection in this mode, a minimum waiting delay can be specified using
Customizing paint queue assignments
An asynchronous layer wrapper always needs a paint queue, which manages one or more image buffers and a painting thread. The paint queue maintains an image buffer for each painting mode (bodies, labels, selection, and so on). It keeps the images updated in its painting thread. In the event dispatch thread (AWT thread, Swing thread), it can then composite the created images, interleaving them with other layers, respecting the proper painting order.
A paint queue manager greatly simplifies the non-trivial task of assigning paint threads and buffers to asynchronously painted layers. All view implementations that support asynchronous painting configure a paint queue manager out of the box.
LuciadLightspeed offers two ways to customize a paint queue manager. The easiest way is to create and set up the default paint
TLcdGXYAsynchronousPaintQueueManager, with a custom paint hint provider. Paint hints are a powerful way to specify how to paint a layer and whether or not the
layer can share a paint queue with its neighboring layers. If you want more flexibility, you can choose to directly subclass
ALcdGXYAsynchronousPaintQueueManager, and manage the assignments yourself.
Observing paint queue assignments
You can visually examine how layers are mapped to paint queues by enabling a debug feature in
The class javadoc explains how to make it use separate colors for asynchronous layers with separate paint queues.
Using paint hints
The default paint queue manager,
TLcdGXYAsynchronousPaintQueueManager, takes its decisions based on paint hints. With a paint hint, you can specify asynchronous painting settings for a specific
layer, such as the thread priority and whether or not labels should be painted asynchronously. A paint hint also allows you
to enforce that certain layers are never painted in the same thread.
The paint queue manager takes the paint hint settings into account to decide whether or not layers can share the same paint queue. For this purpose, a paint hint contains properties that can or cannot be combined. As an example, consider the following layers:
- layer A
a fast vector layer, with important labels
- layer B
another fast vector layer, with unimportant labels
- layer C
a slow raster layer, without labels
Consider the following painting requirements:
fast layers should be rendered in the same paint thread, to minimize overhead
fast and slow layers may not be rendered in the same paint thread, to ensure timely view updates
important labels should always be visible
it does not matter if unimportant labels are always visible or not
This translates into the following paint hints:
- layer A
"painting group" == "fast", "painting mode"== EVERYTHING. In other words: a layer can use the same paint queue as this layer if the latter also has the property value "fast" for the "painting group" and if the "painting mode" property is EVERYTHING.
- layer B
"painting group" == "fast", "painting mode"== EVERYTHING or BODIES_AND_SKIP. In other words, a layer can use the same paint queue as this layer if the latter also has the property value "fast" for the "painting group" and if the "painting mode" property is EVERYTHING or BODIES_AND_SKIP.
- layer C
"painting group" == "slow", "painting mode"== EVERYTHING or BODIES_AND_SKIP. In other words, a layer can use the same paint queue as this layer if the latter also has the property value "slow" for the "painting group" and if the "painting mode" property is BODIES_AND_SKIP or EVERYTHING.
To further illustrate the use of paint hints, the class
PaintTimeBasedAsynchronousPaintQueueManager shows how to assign paint queues with paint hints, based on how long it takes to paint the layers.
Performing your own assignments
ALcdGXYAsynchronousPaintQueueManager you have the opportunity to focus on paint queue assignments without having to worry about whether layers are contiguous,
or how moving layers around affects the assignments.
This paint queue manager divides the view into a number of paint blocks. If a paint block is asynchronous, its layers will always share the same paint queue. The paint queue manager offers methods
to split up and merge paint blocks. To implement your own manager, you only need to implement the
evaluateModifiedPaintBlocks methods and split up and/or merge the relevant paint blocks. As an example, the class
FixedCountPaintQueueManager shows how to implement a manager that uses a fixed number of paint threads.
Providing transparent access to a custom layer interface
If your application contains an extension of
ILcdGXYLayer, and you do not want to use a blocking method to change its non-standard properties, you can extend the layer wrapper to
ILcdGXYLayer extension. The layer wrapper can then offer the same additional properties in its interface. To do this, the layer wrapper
extension must provide an implementation of
ILcdGXYLayerChangeTracker which ensures that the additional properties remain consistent between the layer wrapper and the wrapped layer.
The asynchronous layer wrapper isolates the wrapped layer from the outside world. This way, the program can change properties on the layer wrapper at any time in one thread, for example the Event Dispatch Thread (EDT). At the same time, the paint queue paints the wrapped layer in its painting thread. Neither thread has to wait for the other one. The paint queue synchronizes the properties of the layer wrapper and the properties of the wrapped layer at the appropriate and safe points in time.
The default implementation of
TLcdGXYAsynchronousLayerWrapper, wraps basic implementations of
TLcdGXYLayer. The synchronization of properties between the layer wrapper and the wrapped layer is the responsibility of implementations
Figure 4, “Using an
ILcdGXYLayerChangeTracker” shows a sequence diagram of this mechanism. The
paint method of the paint queue first propagates any changes from the layer wrapper to the wrapped layer. It then queues the actual
operation on the painting thread, so it can return right away. Note that the paint queue can be asked to wait a while for
the asynchronous painting to complete, so that the results are painted immediately. This behavior is set using the method
setSynchronousDelay. After the painting operation is queued, the wrapped layer is painted in an image buffer in the painting thread. When the
painting is done, any changes (for example, label positions) are propagated from the wrapped layer back to the layer wrapper.
Finally, the paint queue invalidates the view, so the newly created image can be blitted to the view on the next repaint.
TLcdGXYAsynchronousLayerWrapper provides an implementation that keeps the basic layer properties consistent.
An example of an asynchronous layer wrapper providing transparent access to another layer interface is
TLcdGXYAsynchronousEditableLabelsLayerWrapper, which can wrap implementations of
ILcdGXYEditableLabelsLayer. In addition to the basic layer properties, it also manages the synchronization of the label positions, from the wrapper
layer to the layer with editable layers, but also the other way round, whenever the label painter has updated the label positions.
Developing and debugging multi-threaded applications can be difficult and error-prone. This list of pointers and tips help you deal with possible trouble during the use and extension of the asynchronous painting API:
TLcdGXYBusyLayerTreeNodeCellRendererto visually examine how layers are mapped to paint queues.
If an asynchronous layer wrapper is not painted asynchronously, you should check if the layer wrapper has a paint queue. Without a paint queue, the layer wrapper just paints synchronously. This only applies to situations in which no paint queue manager is used, which is normally not the case.
If your code ends up in a dead-lock, you should check if the paint methods of your layers and painters do not wait on the event dispatch thread, for example for popping up a menu. The event dispatch thread itself may already be waiting on the painting thread, so the additional wait would result in a dead-lock. Similarly, runnables that are scheduled to be invoked on the painting thread must not wait on the event dispatch thread.
If the asynchronous painting mechanism gets stuck in an infinite painting loop, you should check if you have written any state trackers that propagate properties incorrectly. Notably, they should only propagate actual changes to properties. Furthermore, they must not track and then propagate the changes that they are propagating themselves.
If you see drawing anomalies, synchronous and asynchronous drawing code are probably sharing objects which they should not share. Make sure that your wrapped layer is not referenced by, for example, a controller. Instead, pass the layer wrapper. Another possible cause could be incorrect clone implementations of layer painters and/or the layer pen. Their clones should be sufficiently deep so that they can be safely accessed in another thread.
If you edit your model, make sure that you lock it as described in Editing the model of a wrapped layer.