GSoC 2021: Parallelization of existing modules for GRASS GIS

Title:	Parallelization of existing modules for GRASS GIS
Student Name:	Aaron Saw Min Sern
Organization:	​OSGeo - Open Source Geospatial Foundation
Mentor Name:	Huidae Cho, Vaclav Petras, Māris Nartišs
GSoC proposal:	​View proposal
GitHub Repository:	​View account

Abstract

The current state of OpenMP support for raster modules is limited. Many of such modules can benefit from parallelization. This project aims to provide parallelization to modules chosen based on their frequency of usage and its implementation complexity.

Timeline

Time Period	Milestones
	Tasks	Status
May 17th - June 7th Community Bonding	Introduce myself in dev and SOC mailing lists Get in contact with mentors and discuss project Prepare the wiki page Set up the GitHub repository for project Set up developer environment	Ok Ok Ok Ok Ok
June 7th - June 11th Week 1	Parallelize r.proj Parallelize r.neighbor Parallelize r.univar	-- Pending Ok
June 14th - June 18th Week 2	Parallelize r.cross Parallelize r.blend Parallelize r.composite
June 21st - June 25 Week 3	Parallelize r.mfilters Parallelize r.covar Parallelize r.texture
June 28 - July 2 Week 4	Parallelize r.slope Parallelize r.basins.fill Parallelize r.sunhours Parallelize r.flow
July 5th - July 9th Week 5	Parallelize r.to.*
July 12th - July 16th Week 6: Evaluations	Parallelize r.resamp.*
July 19th - July 23rd Week 7	Parallelize r.resurf.*
July 26th - July 30th Week 8	Parallelize r.random.cells Parallelize r.random.surface Implement support for users to specify number of threads
August 2nd - August 6th Week 9	Parallelize r.li.* Finish documentation and tutorials
August 9th - August 13th Week 10	Finishing up, testing, documentation
August 16th- August 23rd Week 11: Final Evaluation and Code Submission	Submit code and final evaluation

Bonding period report

1) What did I get done this period?

• I have set up a wiki page detailing my project and its progress. (1)
• I have set up my development environment. Here's the link to my repository. (2)
• I have gotten in touch with my mentors, and we are arranging a meeting this week.

2) What do I plan on doing next week?

I will be working on parallelizing 3 modules: r.proj, r.neighbor, r.univar. Based on the results, I will adjust my plans in the future weeks.

3) Am I blocked on anything?

No, it has been good so far.

(1) ​https://trac.osgeo.org/grass/wiki/GSoC/2021/RasterParallelization
(2) ​https://github.com/aaronsms/grass

Weekly reports

Week 1

1) What did I get done this week?

r.univar

• Updated Makefile to include OpenMP dependencies
• Wrote multi-threaded test cases to ensure consistency of the program
• Wrote benchmarking script to measure speedup
• Implemented parallel support
• Drafted the PR of abovementioned changes (1)

r.neighbor

• Investigated Segment library to support random access and write operations

2) What do I plan on doing next week?

The goal is to come out with a design for output-based modules. The next step is to finish the implementation for r.neighbor. Furthermore, I plan to investigate the thread-safety of Raster3D module for pthread implementation of r.mapcalc, which has known issues. (2)

3) Am I blocked on anything?

No, it has been good so far, but I hope to improve on my pace.

Week 2

1) What did I get done this week?

r.univar

• Address changes for the PR (1) , e.g. to use a standard option "nprocs" will now be a parameter for users to indicate the number of threads

r.neighbor

• Write test cases for parallel execution
• Drafted a PR alongside its implementation (2)

r.proj

• Write new test cases for the modules (3)

2) What do I plan on doing next week?

I have managed to come up with a way to parallelize output-based modules like r.neighbor. The idea is to make use of a temporary segment file to allow threads to perform random write operations, which is not possible directly on compressed raster format file without using intermediate cache. With this design in mind, I intend to continue to parallelize similar modules next week. Also, there may be ideas in discussion to encapsulate a benchmarking framework possibly under grass.benchmark as this will be used repeatedly in the future to measure performance.

3) Am I blocked on anything?

No, it has been good so far.

(1) ​https://github.com/OSGeo/grass/pull/1634
(2) ​https://github.com/OSGeo/grass/pull/1654
(3) ​https://github.com/OSGeo/grass/pull/1663
(4) ​https://github.com/OSGeo/grass/pull/1670

Week 3

1) What did I get done this week?

Upon discussion with the mentors, we have decided to explore alternative designs to using Segment library as intermediate output buffer. Specifically, there are two designs in mind, one which simply increases the size of the buffer but does sequential I/O to fill and output from the buffer with intermediate parallel computation, and a more complicated one which tries to eliminate having the threads to wait for the I/O.

2) What do I plan on doing next week?

I plan to finalize the design by this week. (1)

3) Am I blocked on anything?

No, it has been good so far.

(1) ​https://github.com/aaronsms/grass

Final report