regression test

[jive]

During the run up the the latest release; one of the only ways to make progress was to watch people on IRC manually go through the ISO contents and type "ls -a" in each directory.

With that in mind I would like to propose a "regression test" for the osgeo live project. This is based on the cheap and easy way of making regression tests used for Java projects (where you take all the test logs into a single file; and then do a diff from the previous nightly build; and email the diff to the developers list).

Note this is a proper regression test - it does not detect failure - only changes. Even if you just ignore the emails and sort them into a folder - you can still sift through the emails at a later date in order to sort out exactly when things changed.

So here is the plan:

1) do we have a nightly or weekly build resulting in an ISO? 2) write a script to mount the ISO; and perform an "ls -a -R > osgeolive20110923.txt" 3) continue the script to perform a diff between todays contents; and the day before 4) email the diff to the developer list

[hamish]

Hi,

some thoughts--

any file manifest belongs with the ISO and md5s for that at

​http://live.osgeo.org/dev/build/

while potentially a very valuable tool, it doesn't need to be on the disc, doesn't belong in svn, and doesn't really belong at download.osgeo.org either.

sounds like the generating command belongs in a build script somewhere after build_iso.sh? (or before?)

Until we go to automated fortnightly/nightly dev builds of the disc, putting this in a cron job or by-date doesn't gain us anything. It's a single-time post-build task, so gets versioned the same as the build version number.

The manifest file should be bzip2'd; you can compare with bzdiff as you like.

There is no way for the build to know what the previous version was unless we do some svn log+diff command line magic to compare VERSION.txt vs latest_rev - 1. But that's fragile- you might be more interested to compare rc5 vs rc3 if rc4 was a dud.

It should not be automatically posted to the mailing list. I keep a rather tight attachment size limit there on purpose as some subscribers connect over slow bad connections, tiny mailbox sizes, or over expensive cell phone connections, and I would hazard a guess that most of us are already drowning in email. This may not be you, but it is others. automated things like svn commit log messages and automated QA results, belong online or in a second dedicated mailing list IMO.

finally file name should say what's in it, like ${NAME}_${RELEASE}_manifest.txt (I think the "contents" name space is already claimed by overview.rst)

2 or 3c, Hamish

[jive]

Thanks for the small change Hamish. I am really not fussed about the details; more that the visibility would be good (and help with catching changes to annoying things like permissions). I agree that this is best done form a nightly build box; and is not at all interesting to commit into svn.

[darkblueb]

some files generated by

sudo ls -R -a / 

are being accumulated at downloads.osgeo.org / livedvd / must be retrieved by name...

[kalxas]

Since we have a new build system we need to close this issue and move discussion to #1046