r.watershed with MFD

[mmetz]

I open a new ticket for easier tracking of all the bug reports to come...

With the attached patch I want to submit the last (for the time being) modifications to r.watershed. This patch fixes a bug in flag initialization, also present in the original (pre-MFD and pre-faster-search) version. Further on, streams and basins are now reasonably well (IMO) determined for MFD.

Flow accumulation with SFD (single flow direction) is unchanged apart from flow accumulation output now being DCELL instead of CELL.

Flow accumulation with MFD (multiple flow direction) is supported and enabled by default to keep the user interface similar to r.terraflow.

For easy DEMs, e.g. <elevation> in the North Carolina dataset, the results are similar. Differences emerge where MFD kicks in and the flow diverges. Generally, more smaller basins are created with MFD than with SFD keeping the same basin threshold, because diverging flow leads to more cells with flow accumulation above the given threshold.

For difficult DEMs, e.g. <elev_lid972_1m> in the North Carolina dataset, the results are very different. MFD output seems to be more realistic. Here, more smaller basins with unnaturally straight borders are created with SFD. Basins created with MFD seem more realistic and have naturally looking borders, some small basins are created, but no sliver basins like in SFD. Stream segments describe the center of broad streams pretty well, without branching off inside a broad stream.

The results are not perfect, but usable, I think. These are my personal assessments and other people, you, should have a close look at the MFD results and please report any bugs.

Have fun playing around with the convergence factor!

Best,

Markus

[mmetz]

bug fix, MFD basins as good as I get them

[glynn]

Replying to mmetz:

With the attached patch I want to submit the last (for the time being) modifications to r.watershed.

Applied as r34833.

[hamish]

Replying to mmetz:

With the attached patch I want to submit the last (for the time being) modifications to r.watershed. This patch fixes a bug in flag initialization, also present in the original (pre-MFD and pre-faster-search) version.

Hi,

could you isolate that bugfix, i.e. what needs to be backported to 6.4?

Flow accumulation with SFD (single flow direction) is unchanged apart from flow accumulation output now being DCELL instead of CELL.

nice.

Flow accumulation with MFD (multiple flow direction) is supported and enabled by default to keep the user interface similar to r.terraflow.

ok, if there is a major change like that in grass7 it is worth noting clearly in the help page. also the help page needs updating to reflect that it is no longer "much slower than r.terraflow" and the parts referring to the A^{T search algorithm, and any references. I would guess to leave Ehlschlaeger's paper there for the r.watershed ref even if the search algorithm is replaced.}

Have fun playing around with the convergence factor!

...

Hamish

[mmetz]

patch for gras64, flag initialization bugfix

[mmetz]

help page update for MFD

[mmetz]

Replying to hamish:

Replying to mmetz:

With the attached patch I want to submit the last (for the time being) modifications to r.watershed. This patch fixes a bug in flag initialization, also present in the original (pre-MFD and pre-faster-search) version.

could you isolate that bugfix, i.e. what needs to be backported to 6.4?

Attached as r.watershed.bugfix.grass64.patch

Flow accumulation with SFD (single flow direction) is unchanged apart from flow accumulation output now being DCELL instead of CELL.

nice.

Also the code for SFD is unchanged apart from flow accumulation output now being DCELL instead of CELL, maybe I have to emphasize that.

Flow accumulation with MFD (multiple flow direction) is supported and enabled by default to keep the user interface similar to r.terraflow.

ok, if there is a major change like that in grass7 it is worth noting clearly in the help page. also the help page needs updating to reflect that it is no longer "much slower than r.terraflow"

Done in attached patch r.watershed.manpage.patch.

the parts referring to the A^T search algorithm, and any references. I would guess to leave Ehlschlaeger's paper there for the r.watershed ref even if the search algorithm is replaced.

The A^T search algorithm is untouched and in place and also used for MFD. I have already included a reference for MFD, but the actual implementation is a hybrid of MFD and the flow path determined with the A^T search algorithm. The A^T path, the result of the A^T search, is the guidance around which flow is distributed in MFD. Further on, MFD can't get out of depressions and over obstacles, but the A^T path can and still does in MFD. Chuck Ehlschlaeger's paper must stay because also with MFD the whole concept of r.watershed is based on Chuck Ehlschlaeger's paper. The only part that has been replaced is the sorting method for the A^T search to speed up the analysis, the A^T search itself works as before. For MFD, I have not replaced anything, I have only added code, ok, modified existing code to switch between SFD and MFD, but nothing else.

The attached patch named r.watershed.manpage.patch updates the manpage with your suggestions if not already present.

Markus

[hamish]

thanks, all patches applied/merged.

Hamish

[mmetz]

length_slope needs basin threshold

[mmetz]

length_slope needs basin threshold

[mmetz]

the two attached patches

r.watershed.64.length.slope.patch

and

r.watershed.70.length.slope.patch

check whether a basin threshold has been specified when length.slope (length_slope in 70) is specified as output. This has been a requirement in the code before, but wasn't enforced and reported to the user.

Markus M

PS: Please leave the ticket open, more fixes to come for MFD mode.

[mmetz]

A new patch against trunk r35238 is attached, named r.watershed.20090107.patch

This patch includes the following changes:

• MFD flow accumulation is slightly improved.

• MFD streams and basins are now equal to or better than SFD streams and basins (they look more natural, no slivers or rectangles). Striking differences are visible with elev_lid792_1m (multiplied by 1000 to get mm accuracy) in nc_spm_08.

• Segmented mode is slightly faster, more noticeable with larger regions.

• In segmented mode, the dseg routines for DCELL output maps were actually writing out CELL maps, not DCELL maps, changed to DCELL maps. DCELL_TYPE is required for LS and S factor for USLE and for flow accumulation.

• Improved color rules for flow accumulation based on standard deviation and log transformation, visual output would be obsolete now. BTW, in visual output all cells with negative flow accumulation (offmap inflow) are set to zero. That's an old, existing feature.

• In segmented mode, G_percent() in length slope determination is now counting up, no longer counting down (was a FIXME).

Can you developers please test this patch? Thanks!

Happy New Year,

Markus M

[mmetz]

MFD: improved flow accumulation, streams and basins plus various fixes

[mmetz]

I have submitted a new version of r.watershed to trunk with various changes

MFD is fully supported: drainage directions, stream segments, basins, half-basins, RUSLE LS and S factors are all in accordance with MFD flow accumulation.

The module is slightly faster and uses a bit less memory. See updated documentation to estimate memory requirements for the ram (all in memory) mode (max 31 MB of RAM for 1 million cells).

It is no longer necessary to create a MASK for cells with unknown elevation (NULL values). All cells with unknown elevation (masked or NULL values) are skipped. Zero and negative values are valid elevation values.

The color rules generated by the module for flow accumulation have worked for all tests so far and allow visual inspection without manually assigning a color table. I'm using standard deviation and log transform, based on Hamish's (I think) suggestions. Visual output is obsolete but left for comparison.

The output drainage direction is slightly changed in case an input map with real depressions was supplied. Previously, real depressions had a drainage direction of -1, but -1 was and is a valid drainage direction indicating a drainage of 45 degree CCW from East towards a cell with unknown elevation. Therefore real depressions have now a drainage direction of 0. As before, if a cell drains due East the drainage direction is 360 degrees, 0 (zero) as drainage direction was unused.

RUSLE LS and S factor output maps are now of type DCELL without multiplication by 100. Both factors have always been processed as type double, previously the output maps for the two factors were written out by typecasting from (DCELL * 100) to CELL.

Please read the updated documentation before using the new module.

Best regards,

Markus M

[mmetz]

Helena Mitasova and Michael Barton asked previously about backporting r.watershed with MFD to grass6. Now that develbranch_6 stays as version 6.5.svn, does someone have objections against backporting r.watershed from trunk to develbranch_6?

It would be backported with the following modifications for backwards compatibility:

disable MFD by default, it will be enabled with a new -f flag. I don't want to (ab)use the -s flag of trunk because it has the opposite meaning. The -s flag will be removed.

RUSLE LS and S factors will again be multiplied by 100, as before. The output raster type of the LS and S factors has to be changed to DCELL though, otherwise MFD won't work, because flow accumulation needs to be of type DCELL for MFD, i.e. always, and the segmented mode uses the same function for writing out any variable of type DCELL. Previously, LS and S factor were stored as type DCELL, but both in segmented mode and in ram mode written out as type (CELL) (factor * 100). This change from CELL to DCELL can not be reverted, at the same time supporting flow accumulation output as type DCELL, without a lot of rewriting, actually more rewriting than was necessary to implement MFD support. The change from CELL to DCELL for flow accumulation and the RUSLE factors will hopefully not break scripts.

I would need some help on how to properly backport the various changes from trunk to devbr6. According to the svn howto, svn merge backports one particular change if I got that right.

Markus M

[mmetz]

backported to debr6 in r35905 with full backwards compatibility, closing ticket.