wiki:GSoC/2021

Version 4 (modified by wenzeslaus, 4 years ago) ( diff )

add testing framework

Grasslogo vector small.png @ 500px-GSoC2016Logo.jpg​ @ OSGeo logo

GRASS Google Summer of Code 2021

Table of Contents

  1. About
  2. Ideas
    1. Title of idea
    2. Seamless integration of GRASS GIS and Jupyter Notebooks
    3. Integrate testing framework with GitHub Actions
    4. Improved color management in GRASS GIS
    5. Enhance 3D rendering capabilities in GRASS GIS
    6. GRASS GUI: Single window layout
    7. New easy-to-use CLI and API for GRASS GIS
  3. Tips for students

About

Ideas

Post your ideas here or to the ​grass-dev mailing list if you want to discuss them more. To edit this wiki, you need to ​login with an ​OSGeo Userid; read also some help for using Trac.

If you are a student you can suggest a new idea or pick up an existing one in any case write about it to ​grass-dev mailing list.

You are invited as well to have a close look at (and re-suggest!) ideas from previous years (​2007, ​2008, ​2009, ​2010, ​2011, ​2012, ​2013, 2014, 2015, 2016, 2017, 2018, 2019, 2020) which have not yet been implemented. You can also look at accepted GRASS GSoC projects from previous years for an idea of scope.

Include "GRASS GIS" in the title of our idea to easily distinguish ideas and projects inside OSGeo.

Some bigger ideas may have their own pages, so you can link them here. The pages can be either independent if the page already exists (e.g. wxGUIDevelopment/SingleWindow), or more preferably subpages of this page if the idea is (re-)developed for this GSoC. In the later case, use the word "idea" in the page name to distinguish the idea page (e.g. GSoC/2021/CoolGRASSProjectIdea) from the possible student project page (e.g. GSoC/2021/CoolGRASSProject).

Title of idea

Description here

  • Requirements:
  • Mentor:
  • Proposed by:
  • Rating:
  • Expected Outcomes:
  • Test of skills:
  • Other:

Seamless integration of GRASS GIS and Jupyter Notebooks

  • Start of GRASS GIS inside a script or a notebook is currently too cumbersome.
  • Maps do not allow panning and other interactions.
  • There are examples how to use Jupyter Notebooks with GRASS GIS, but they need to be improved.
  • Requirements: Basic knowledge of GRASS GIS scripting and IPython/Jupyter Notebooks, Python, JavaScript
  • Mentor: Vaclav Petras
  • Co-mentor:
  • Rating: easy
  • Expected Outcomes: IPython/Jupyter Notebooks related to grass; some new functions in grass library to show data/tables on IPython/Jupyter Notebooks
  • Test of skills: Write a testsuite for a GRASS GIS command, more info here

Integrate testing framework with GitHub Actions

  • The testing framework was written before migration to Git/GitHub and when long free runs in 3rd party services were unthinkable. Additionally, different goals overall were prioritized.
  • An update is needed to reflect that tests are used to evaluate pull requests (PRs).
  • Most of the updates should be generic enough to integrate well with any CI environment. However, currently GRASS GIS is using GitHub Actions which open the possibility to make use of specific features there, e.g., upload artifacts for failed tests or do a quick test only of files changed in the PR.
  • Possibly, use pytest (following example of GDAL and Numpy) instead of Python unittest (some parts of grass.gunittest might be replaced by or ported to pytest).
  • Current tests don't accommodate well tests of the main GRASS executable (which is now more prominent also because grass ... --exec).
  • Both preserving and removing of existing HTML reporting functionality are valid options.
  • Requirements: Python, GitHub Actions (YAML configuration)
  • Mentor: Vaclav Petras
  • Proposed by: Vaclav Petras
  • Rating: medium
  • Expected Outcomes: Effective reporting of test results in pull requests
  • Test of skills: Fix failing tests and/or write new tests (more is better). Alternatively, addressing a smaller problem in the testing framework is a good task, too.

Improved color management in GRASS GIS

Current color management of raster and vector maps has several issues:

  • Interactive editing of colors is rather limited: #3370
  • We can't easily specify discrete color intervals (e.g. ​this ramp) for floating point data unless we reclass first, legend does not display that correctly
  • ​and others...

A new color editing widget could be developed that would be integrated in r.colors, r3.colors, v.colors, t.rast.colors. It would allow simple editing of breaks (manual and automatic). Better support for discrete color tables for floating point data needs to be developed.

  • Requirements: Python, wxPython, possibly some C
  • Mentor: Anna Petrasova, Vaclav Petras
  • Proposed by: Anna Petrasova
  • Rating: medium
  • Expected Outcomes: More user-friendly color management
  • Test of skills: ​and others...

Enhance 3D rendering capabilities in GRASS GIS

Current 3D rendering capabilities in GRASS GIS (called NVIZ) are quite powerful, but many important features are missing. The current implementation is rather old and needs to be updated with more recent technologies. The following points (fixes and new features) should be addressed:

  • fix rendering raster and vector data with transparency
  • faster rendering of point clouds
  • implement text rendering (for scale bars for example)
  • review and fix problems with current rendering of 3D vectors
  • possible new features would include volume rendering (ray casting for example), so far we can visualize only isosurfaces or slices of 3D rasters
  • adding axes
  • Requirements: C, OpenGL
  • Mentor: Anna Petrasova
  • Co-mentor: Vaclav Petras
  • Proposed by: Anna Petrasova
  • Rating: medium
  • Expected Outcomes: reliable on-screen and off-screen 3D rendering on most platforms
  • Test of skills: #2076, #3743

GRASS GUI: Single window layout

Currently, GRASS GIS GUI (wxGUI) has one Layer Manager window and one or more Map Display windows. This multiple window layout can sometimes be problematic and is not common. This project would try to develop an optional single window layout, similarly to ​GIMP. This project would include:

  • detailed design of the behavior of current components + mockups
  • refactoring some of the wxGUI components to untangle them and make them reusable
  • implementing the actual layout and its behavior while keeping the multiple window layout working
  • Requirements: Python, wxPython
  • Mentor: Anna Petrasova, Vaclav Petras
  • Proposed by: Anna Petrasova
  • Rating: medium
  • Expected Outcomes: enable switching between single- and multiple-window layout
  • Test of skills: Develop a simple wxPython application, which would allow to switch between these two modes. This should demonstrate student knows wxPython and general GUI design.

New easy-to-use CLI and API for GRASS GIS

  • Requirements:
    • Language: Python
    • Proposal: Student needs to show sufficient understanding of the GRASS GIS Database structure and significantly extend on text below in terms of more concrete formulation of ideas and identification of missing and existing parts.
  • Mentors: Vaclav Petras
  • Co-mentors:
  • Proposed by: Vaclav Petras
  • TL;DR: Make running of GRASS GIS modules as easy as it is to run GDAL commands.
    • grass run r.slope.aspect elevation=elevation.tiff slope=slope.tiff aspect=aspect.tiff
    • CLI like GDAL has.
    • No GRASS Database, Location, Mapset to deal with.
    • No import, export from user perspective.
    • Reasonable defaults for things like region.
    • CLI and API still allows user to specify any of the above.
  • GRASS GIS requires GRASS GIS Database, Location and Mapset to be set up to maintain data consistency, efficiency and security. Unfortunately, this is cumbersome when GRASS GIS is not the primary tool user is using.
  • There are different ways for calling GRASS modules without starting iterative GRASS session:
    • Modules executed with the --exec interface (see the grass7 manual page >=7.2)
    • GRASS_BATCH_JOB: same as newer --exec but through environmental variable and more limited
    • Use grass.script.setup package from GRASS GIS (requires boilerplate to add the packages on path first)
    • Use the standalone grass_session package (new, see ​here)
    • Set up environmental variables and "RC file" yourself (the classic method).
  • None of these allows the user to skip the database setup phase. This leads to the need for constant reimplementing of setup, import and export steps in various software and environments including user scripts (in Bash, Python, R), QGIS Processing, gvSIG/SEXTANTE, uDig/JGrassTools, and all the web/server/cloud tools and applications which use GRASS GIS as a processing backend (e.g. PyWPS server).
  • GRASS GIS itself can make it easier for the callers (at least in most cases) by implementing an interface which would allow to use GRASS GIS modules without explicit dealing with GRASS GIS database.
  • The command line call using the proposed interface would look like these: 
grass run r.lake elevation=some/file.tiff water_level=10 lake=some/new/file.tiff coordinates=100,520
grass run r.slope.aspect elevation=file://.../elevation.tiff aspect=file://...aspect.tiff
grass run r.slope.aspect elevation=https://.../elevation.tiff.zip aspect=file://...aspect.tiff
  • Basic execution phases:
    • The grass command would have to parse the command line, compare it with the module XML interface description, find the files which should be maps (either using file:// and ideally anything else), potentially download and uncompress, and import (or link) them, and then call the actual command (GRASS module).
      • The input maps could be linked (external) rather than imported (except for the cases when projection differs) which should be faster than import.
      • Doing the work in GRASS rather than in the other software would allow GRASS to make the decision about the details, for example the data exchange (r.external vs r.import vs r.in.gdal - see ​comment from MarkusN for QGIS Processing issue or ​mailing list).
    • GRASS Database would be created with an appropriate Location (projection based on input files or additional CLI input).
      • The GRASS GIS Database, Location and Mapset should be created on the fly and deleted afterwards (the .grassrc wouldn't be used).
    • Computational region would be set based on input file(s) or additional CLI input.
    • Module execution.
    • The output maps could be be also linked (e.g. r.external.out) with projection same as input which is should be faster then export.
      • Ideally export (as well as import) should work also with PostGIS and databases provided through GDAL/OGR.
  • Proposal should discuss and address how advanced things such raster algebra, multi-map inputs and outputs, temporal framework, cartography and visualization tools will work (or what are the limits).
  • Use should be able to always specify the details manually: 
grass run --mapset=/some/directory/grassdata/ncspm/practice1 r.lake elevation=some/file.tiff ...
grass run --region="s=55600 n=60500..." --mask=some/mask.tiff r.lake elevation=some/file.tiff ...
grass run --crs=EPSG:3358 --mask=some/mask.tiff r.lake elevation=some/file.tiff ...
grass run --use=some/file_a.tiff --get=some/file_b.tiff r.slope.aspect elevation=file_a aspect=file_b
  • The system behind the interface will be inherently fragile, so it is necessary to write large amount of tests which would check different combinations of data types and projections.
  • All the underlying code is expected to be in Python, so the project should involve also creation of Python API on the way.
  • Bonus tasks:
    • Making this work for the GUI in the same way. It is expected that this would work for any g.gui.* modules too but implementing similar mechanism also for module dialogs is more work (but some basic implementation might be quite straightforward).
    • Making this connected to the standalone grass_session package.
    • Generalization of the API, so that it incorporates also the concept of remote sessions (see e.g. ​g.remote on GitHub)
  • Current GRASS code involved:
  • See also:
  • Test and training tasks:
    • Extend --exec functionality:
      • Add --clean (current default) and --no-clean which say if --exec should clean the .tmp directory in the Mapset (for parallel execution). (See also #3537.)
      • Add --lock (current default) and --no-lock which say if --exec should lock the Mapset (for parallel execution). See also #2685 and the -f flag.
      • Add --region to set a temporary computational region for the execution, e.g. --region="raster=raster_name"
      • Add --import-raster=some/file.tiff which imports (r.in.gdal or r.import) a raster file (same for vector).
      • Add --link-raster=some/file.tiff which links (r.external) a raster file (same for vector).
      • Add --export-raster=some/file.tiff which exports (e.g. r.out.gdal) a raster file (same for vector).
      • Add --link-output-raster=some/file.tiff which creates (r.external.out) a new (output) raster file (same for vector).
    • Add features to grass executable interface:
      • Make it possible to associate *.gxw files with grass executable (#1204) or at least add --gui-workspace or preferably just recognize it in the command line (distinguish it from database/location/mapset).
    • Solve one of the tickets linked above.

Tips for students

  • If you have your own ideas we encourage you to propose them. Explain them on the ​grass-dev mailing list.
  • If you like some idea here or from previous yeas, write about it on ​grass-dev mailing list and any ideas of your own which could improve it.
  • Follow some good practices in your ideas and proposals:
    • Stress why the project would be useful.
    • Show that you know how you will proceed. That is, make sure that you can demonstrate that the proposal is feasible in the given time frame.
    • Be specific in the implementation (or at least as specific as you can).
    • Explain what the final product will look like and how it will work. Perhaps you can add some drawings or mock-ups. (here in a wiki page)
    • Explain how the idea relates to existing GRASS GIS functions, features, and needs.
    • Do not include steps such as "install GRASS", "compile GRASS libraries (on my machine)", "read about the API". You should do this before applying to GSoC.
  • Compile GRASS GIS 7 from source and prepare environment for development:
  • Prove your worth by being active on the GRASS mailing lists (​grass-user, ​grass-dev), fix some ​bugs, and/or implement some (smaller) features, or write some (simpler) GRASS module, and post it to mailing list. There's no better way to demonstrate your willingness and abilities. You should start even before you apply to GSoC.
  • Also note that fixing existing bugs and/or implementing enhancements will be a part of student evaluation.
  • Every year GRASS GIS hopes to participate and participates in GSoC as part of the ​OSGeo Foundation's GSoC program umbrella. See the official OSGeo template for application details and other important information at the ​OSGeo GSoc Ideas page.
Note: See TracWiki for help on using the wiki.