GRASS GIS 8 in OSGeo4W NG

[sbl]

The GRASS GIS devs are preparing the release of GRASS GIS 8.

With the upcoming new version in mind, I would like to discuss how we can get it into OSGeo4W.

Personally, I could imagine packages similar to QGIS (but not as many):

grass (refering to the current stable (point) release ) grass-rel (nightly build of the release branch) grass-ltr (refering to the legacy release, we do not have a concept of LTR/LTS right now (would become 7.8 once 8.0 is released)) grass-dev (nightly build of the develpment branch)

But here we are open for discussions, and solutions like with GRASS GIS 6 and GRASS GIS 7 are of course thinkable as well!

I am willing to contribute (in fact I tried to create a grass-dev package here: ​https://github.com/ninsbl/OSGeo4W/tree/grass_dev_workflows), where I tried to build the package with GH workflows. So, I would try to prepare PRs. Unfortunately, I cannot figure out what the entrypoint for packaging is in ​https://github.com/jef-n/OSGeo4W (I am more or less bound to GH actions), so I would need some help to get started.

Hope we can solve that together somehow...

[jef]

Unfortunately there is no entry point.

The original idea was to have some trigger on commit that figures out which packages need rebuilds (changed packages + reverse dependencies) and trigger their package.sh in order.

But I never got to that, as I cannot get QGIS to build (let alone Qt) using github actions because of disk space constraints. So I still have to build packages "outside".

Otherwise entry points would be the individual package.sh, which:

• declare build dependencies, that are fetched in startlog,
• fetch the sources
• build
• package in a local repository, which are finally uploaded in endlog

e.g. ​GRASS package.sh

[sbl]

Thanks for your reply. For starters I tried (in a Github action), to:

1. Download dependencies from OSGeo4W (so they live in /c/OSGeo4W/,
2. Setting the current working directory to where the package.sh lives (src/grass/osgeo4w) and run it with bash.

However, although the relative path should be correct, build_helpers was not found, and after hardcoding the path to build_helpers, build_helpers was sourced, but then genini fails:

/d/a/OSGeo4W/OSGeo4W/scripts/genini: can't open x86_64/setup.ini.tmp - No such file or directory
mv: cannot stat 'x86_64/setup.ini.tmp': No such file or directory
/d/a/OSGeo4W/OSGeo4W/scripts/build-helpers: line 206: x86_64/setup.ini.bz2: No such file or directory

The attempt to setup the GH action is here: ​https://github.com/ninsbl/OSGeo4W/tree/grass_dev_workflows and the results (error messages) here: ​https://github.com/ninsbl/OSGeo4W/runs/3620990391?check_suite_focus=true

My feeling is that something needs to be run before package.sh (maybe some environment variables need to be set) or that the script needs to be called from a different place than where it lives. But I have no idea where I could find information on that.

If I could get that to work, I could contribute recipies for GRASS packages (PRs that are tested on GH), and probably even more important with regards to OSGeo4W integration, I could try to integrate the OSGeo4W build procedure as the basis for the CI / unittests for GRASS GIS, so new commits are tested with regards to effects on OSGeo4W builds...

So, e.g. any hint on how to call package.sh and pre-requisits to do so successfully, would be a great help and very much appreciated. Thanks.

[jef]

Replying to sbl:

Thanks for your reply. For starters I tried (in a Github action), to:

1. Download dependencies from OSGeo4W (so they live in /c/OSGeo4W/,

That's not necessary. Each package installs it's own copy of OSGeo4W dependencies in src/$P/osgeo4w/osgeo4W.

2. Setting the current working directory to where the package.sh lives (src/grass/osgeo4w) and run it with bash.

But src/grass/osgeo4w/package.sh already exists!?

However, although the relative path should be correct, build_helpers was not found, and after hardcoding the path to build_helpers, build_helpers was sourced, but then genini fails:

/d/a/OSGeo4W/OSGeo4W/scripts/genini: can't open x86_64/setup.ini.tmp - No such file or directory
mv: cannot stat 'x86_64/setup.ini.tmp': No such file or directory
/d/a/OSGeo4W/OSGeo4W/scripts/build-helpers: line 206: x86_64/setup.ini.bz2: No such file or directory

Ok, x86_64/release was apparently not created automatically. IIRC this wasn't ever run on github, because QGIS isn't buildable anyway - after hours it would run out of space…

[sbl]

Thank, @jef for following up. So, I will skip the OSGeo4W download.

In the github action the OSGeo4W repo is pulled to the build server, so the entire repo content is present at /d/a/OSGeo4W/OSGeo4W

Will try with creating x86_64/release in advance. Is it correct to create it in: src/$P/osgeo4w/ and call package.sh from there or where should the working directory be?

[jef]

Replying to sbl:

Will try with creating x86_64/release in advance. Is it correct to create it in: src/$P/osgeo4w/ and call package.sh from there or where should the working directory be?

​https://github.com/jef-n/OSGeo4W/commit/bdd21f516094f96fc8f3bcaa0b04bc85b7a62cb0

[sbl]

Thanks, jef. This seems to have been a missing link for me. I enccountered some other minor issues with running this in a GH action, but most of them I managed to iron out.

Currently, I encounter issues with dumpbin.exe but that is probably just a misconfigured PATH and versions beeing different between GH actions and (dumpbin -exports does not work while dumpbin /EXPORTS does). Testing right now if my changes make it run through...

Will try to cleanup once this works properly and then create a PR.

Thanks again with your support on this...

[sbl]

Note that RC1 for GRASS GIS 8 was released end of 2021:

​https://github.com/OSGeo/grass/releases/tag/8.0.0RC1

[sbl]

A new, RC2 for GRASS GIS 8 has just been released: ​https://github.com/OSGeo/grass/releases/tag/8.0.0RC2

[jef]

grass8 was added a while ago (currently at 8.0.2-1)

[sbl]

Good to have 8 and 7 available. I understand that nightly builds are a bit of a hassle (esp. if downstream dependencies would have to build against it as well). Yet, I would still love to see a nightly build of the latest GRASS GIS development version (and maybe the current stable version) to simplify real-life user tests... Updating a GRASS GIS package is much faster than downloading a full standalone installer...

[jef]

Replying to sbl:

I understand that nightly builds are a bit of a hassle (esp. if downstream dependencies would have to build against it as well). Yet, I would still love to see a nightly build of the latest GRASS GIS development version (and maybe the current stable version) to simplify real-life user tests...

nightly of GRASS also inplace (grass-dev)

[sbl]

Great! Thanks so much!