Map rendering

This page is about rendering in wxGUI. It contains proposals how to make it better. Proposals can be incomplete but complete proposals are better. It it not about general map rendering in GRASS, however it may obviously interfere.

Introduction

• affects both the main GUI and the wx-based d.mon
• probably a lot of I/O operations makes current system slow
• the generated PPM files can have several MBs
• for d.mon there are also some questions about how zooming and region handling should work (may or may not be connected to rendering -- display vs. computation region)
• specifics of d.* modules other than d.rast and d.vect makes things more complicated
• very different user interfaces for the main GUI and d.mon can make things more complicated
• there are also things such as WMS services

Related pages

Related tickets:

• #1719 GRASS 7 Monitor command line support
• #1926 g.gui.animation: parallel rendering with d.rast and d.vect

Related discussion on mailing list:

• ​Speed map display in wxGUI on bigger monitors (​at Nabble)
• ​GRASS_RENDER_IMMEDIATE in GRASS7 (​at Nabble)

Related changesets:

• r54465 fix for not updating statusbar (possibly slowed down the rendering)
• r52436 wxGUI: delete unused wx.Image instances when rendering
• r48040 show 'busy' mouse cursor while rendering

General problems affecting current wxGUI

• it is in Python and simple loop is always slower in Python then in C/C++
• loading of dynamic (wxWidgets, wxPython) libs and modules always takes some time, when you open display/GUI for the first time (when I'm opening the second display it is much faster)
• rendering to files and then displaying this files on screen is slower than direct drawing of data to screen

About the last point, we are using rendering to files because displaying/drawing of maps is done by modules, not library functions (the functionality is in the library). Moreover, it is not safe to call GRASS library functions from GUI since they call G_fatal_error() which calls exit(), so on error the whole GUI ends (without an error because stderr is written into GUI console which ends too).

Currently used methods

• wxGUI renders to uncompressed PPM and uses g.pnmcomp
• wxWidgets-based xganim reads a raster file directly and stores data in wxImage
  • it is fast in C++ but to do the same in Python you have to use some NumPy magic to make it at least comparably fast (the reading and displaying data is done in loops)
  • G_fatal_error() and exit() are still here
  • in order to provide full drawing capabilities code from d.* modules would have to be moved to library
• wxPython-based g.gui.animation uses d.* with process Queue
  • d.* commands enables to render both vector and raster
  • process Queue makes it faster (on multicore) even if the images are written to disk
• older wxPython-based g.gui.animation used r.out.ppm and NumPy
  • NumPy syntax is unreadable in this case
  • Python loop was just so slow to be used
• wxNVIZ uses OpenGL
  • the rendering library is in C
  • in wxPython GUI used through ctypes
  • data are accessed through the standard GRASS GIS library functions (thus G_fatal_error() and exit() are there)
  • TODO: provide more details if necessary, or delete this comment
• ximgview (C application which uses X)
  • reads BMP images generated by PNG or cairo driver and uses GRASS_PNG_MAPPED=TRUE to get constant file size
  • TODO: only documentation was used, provide more details from code, probably same as wxWidgets version
• wximgview (C++ application which uses wxWidgets)
  • reads BMP images generated by PNG or cairo driver and uses GRASS_PNG_MAPPED=TRUE to get constant file size
  • uses time to look to the file then read the file in for loop

Proposal: Do the composition in the wxGUI

The wxPython GUI should really be doing its own compositing rather than relying upon g.pnmcomp.

The original reason for composition by a module was that Tk doesn't support PNG. That isn't relevant to the wxPython GUI.

Can PIL be used for this? It is a new dependency only partially since currently Cartographic Composer preview depends on it.

(glynn, ​http://lists.osgeo.org/pipermail/grass-dev/2012-August/059188.html, with some comments)

Proposal: Use compressed files to create smaller files

change the PPM files to something compressed (see ML)

​http://lists.osgeo.org/pipermail/grass-dev/2012-August/059099.html

Then I got impression that there is some issue with PNG driver (if we want to use it instead of PPM) and also that cairo driver could help but it is not ready.

Experimental code: PNG rendering and composing in Python

Experimental code to test PNG rendering and composing in Python: attachment:png_rendering_pil_composition.diff:ticket:1719​

In the diff you can see some changes which are necessary for rendering PNG (instead of PPM) and composing of images in Python using PIL (instead of g.pnmcomp).

It is not completely working code. It is only experiment to see if this can be faster.

The answer is no. I don't have any numbers but user experience is completely the same.

Files are much smaller and there is one file less (thanks to composing in Python) but during zooming/panning the most of the time is spent with disk IO (tested on Ubuntu 10.04). Zooming and panning require re-executing the d.* commands to generate new images.

The downside of the compressed format is that the d.* command will take longer as it has to compress the output file.

If desired, we could have a single file without the compression overhead by using the PNG writer from lib/pngdriver, which allows the compression level to be set via the GRASS_PNG_COMPRESSION environment variable. (Move this paragraph to some proposal?)

(wenzeslaus, annakrat, comment:14:ticked:1719, 2012-09-07, glynn comments included)

Proposal: Output binary data from d.* modules

We can output binary data to stdout and read them in Python directly. This would avoid disk IO. In other words stdout instead would replace usage of a file.

That requires either storing all layers in memory, regardless of whether or not they are displayed, or re-generating layers if they are disabled then re-enabled. It also eliminates the possibility of implementing a decent caching mechanism (i.e. being able to undo zoom/pan operations by re-using the previous images rather than having to re-generate them).

And using pipes may be slower than disk (if the Python side is using select(), there will be a context switch for each pipe-buffer-size of data).

(wenzeslaus, comment:14:ticked:1719, 2012-09-07, glynn comments included)

Proposal: Use X Pixmaps

In theory, the fastest solution should be to use the Cairo driver with output to X Pixmaps. The d.* modules generate output in video memory, and may be hardware accelerated. Compositing occurs entirely in video memory, and may be hardware accelerated. The main unknowns are how hard wxWidgets makes this, and whether something similar can be achieved on other platforms.

Details and disadvantages

The cairo driver can render to an X Pixmap. These can then be composited via g.cairocomp. Rendering and compositing may be hardware-accelerated, and the resulting raster data will never leave video memory.

However:

• Requires X. I don't think that there's any equivalent on Windows. X is only reliably available on Linux. Not really an option for Windows or Mac.
• It relies upon the fact that an X client can create resources (e.g. Pixmaps) on the server which are retained after the program terminates.
• Requires that cairo was built with XRender support (although that's almost certain on X; cairo was created largely to facilitate the use of XRender).
• Can wxWidgets use an X Pixmap directly (i.e. without copying the contents to client memory and back again)?
• Leaving "stale" images in the X server's memory is even worse than leaving stale PPM files in the temp directory. Even keeping all of the "live" images in the X server's memory may be an issue on some systems.

wxPython implementation

In wxWidgets there is wxBitmap (​wxPython, ​wxPython Phoenix, ​wxWidgets). This wxBitmap can be directly drawn on some wx widget/window. This operation is fast. The data for an X Pixmap resides within the X server. On the client side, a Pixmap is just an XID (a 32-bit integer identifying a server-side resource). For X11-based versions of wxWidgets, a wxBitmap probably has an associated Pixmap, but I can't see any way to get at it, or to create a wxBitmap for an existing Pixmap.

Rendering of the image

If GRASS_PNGFILE ends in ".xid", the cairo driver will use the XRender back-end, and will write the XID of the underlying Pixmap to the file specified by $GRASS_PNGFILE. It will also call XSetCloseDownMode(dpy, RetainTemporary), which results in server-side resource being retained when the X connection is closed.

Another program can read the XID from this file, and use it with any Xlib function expecting a Pixmap or Drawable argument. E.g. g.cairocomp uses cairo_xlib_surface_create_with_xrender_format(), which takes the XID of an existing Drawable as an argument (​link to doc).

Thus, the d.* module creates the Pixmap on the X server and leaves it there for other programs to use.

(glynn, comment:11:ticked:1719, 2012-09-05, with some other comments)

Proposal: Use different rendering engine for GUI and for monitors

Use different rendering engine for the main (big) GUI and for (wx) monitors. This engine could use library functions and can call exit() because it would end only one monitor, not the whole GUI (and the error output is written into the command line).

But there are two problems. The first is mirroring of display architecture provided by modules and the second is maintenance of additional code (but this can be only minor issue if it is well designed). Basically it would be necessary to move all display (rendering) code into the library and use this library for d.* modules and wx monitors. However, it is not clear if ctypes would be enough to implement it in wxPython and thus (possibly) part of the code should be in wxWidgets (which could be easier with wxPython Phoenix).

The idea of two different approaches for main GUI and monitors is not new. The current wxGUI (at least before r55230) uses different approach for layer rendering (the difference is the number of help files and the opacity handling).

(#1719, 2012-06-04)

Proposal: Directly access data with NumPy

With the help of GRASS Python modules, it easy to pull chunks from raster data as NumPy array. A new rendering model (in-memory) can be added to wxGUI which can directly display a NumPy array as wx.Image. This will obviously increase the performance of rendering layers because the overhead of reading and writing data to file (PNG adds a compression during packing and unpacking) can be saved. Also the extent based rendering will allow to read portion of raster image than whole dataset. Also a tile based technique can be added to this which improves the rendering performance further.

Basically this will do what d.rast or generally speaking d.* modules do. I agree adding other modules such as d.legend etc is a massive work. But compared to dealing with large dataset I think its worth it.

This is just a brief of what I have. I can provide further details if anyone is interested. (rashadkm, 2013-03-29)

Proposal: Use memory-mapped files or shared memory for output of d.* modules

TODO: Comments not included because this requires further analyses anyway.

(ychemin, 2013-04-15)