discussion/planning prior to Open Source Report Card forms

[tomroche]

summary

This page is for refining a plan to upgrade the ​OSGeo MediaWiki (MW) to provide UI and backing store for Open Source Report Cards (à la ​Google's). Let's discuss here what we believe needs done for this task, then spinoff separate ticket(s) for the plan and any major subtasks that emerge from the discussion.

Apologies for the lack of external links below, but Trac is spam-bucketing me for providing too many :-(

details

Suchith Anand started ​a thread requesting a way for OSGeo orgs to publish their Open Source Report Cards. After some discussion, several of us concurred that The Way To Do This was via the OSGeo MW, so I started a ​more targeted subthread, which now migrates to this ticket.

I make an initial proposal below: feel free to append alternatives. This task feels straightforward (famous last words :-) so I'm guessing we can have this discussion wrapped before 16 Nov 2016 (i.e., a week from when most folks will actually see this ticket or the email announcing it) ... but ICBW.

Roche proposal

Following is a first approximation: see tasks expansions in following subheads. Note also that some of these tasks (notably prototyping and maintenance, and prototype development and testing) can run in parallel.

1. Setup initial development on one of the ​free public Page Forms hosts (aka freehost).
2. Prototype task datastructures and forms on freehost.
3. Test prototype with toy data on freehost.
4. Alpha-test prototype with small group of OSGeo volunteers.
5. Do deferred maintenance on OSGeo wikihost.
6. Transfer alpha-tested prototype to wikihost.
7. Beta-test with larger group of OSGeo volunteers.
8. Go live/global.

setup accounts on free public Page Forms host

One or more of those freehosts (e.g., Referata) should already be up-to-date on the platform components (step#=5 below), which will allow us to continue to defer already-deferred maintenance, or (preferred) do that in parallel. Folks on this ticket (aka the devgroup) can get accounts on the freehost and get to work in a space completely isolated from the current/working OSGeo MW.

prototype task datastructures and forms

... on the freehost. Basically just like steps 1-5 in the ​Page Forms example, except that our usecase is different: from Suchith Anand's post, we will need instead (e.g.)

• Name of Organisation
• Name of Contact person
• Email of contact person
• Website
• Country
• Year
• Which open source projects have your organisation contributed with details/links etc
• Which open source codesprints, conferences, events have you supported this year
• Plans for next year

test prototype on freehost

By devgroup, with toy data, can be done in parallel with previous subtask. Basically, test that we can output via query the same data that we input via webforms. Should develop some testcases while we're at it.

alpha-test prototype on freehost

At this point, prototype needs to get out of the devgroup bubble and "get its tires kicked" by a small group of OSGeo volunteers (who will need to get freehost accounts). Incorporate their UI and query criticisms, loop back to step#=2 until the alpha-testers are satisfied.

update OSGeo wikihost

Get the OSGeo wikihost up-to-date with latest (at least, security updates) for components important to this task, notably MySQL, PHP, MW, and the MW extensions Semantic MediaWiki (SMW) and Page Forms (PF).

transfer alpha-tested prototype to wikihost

Should be straightforward--just some grunt work.

beta-test

... with larger group of OSGeo volunteers, or restrict to OSGeo members.

deploy

... and maintain, fix bugs, accommodate FRs, etc.

[tomroche]

I got a Referata account (plus a scratchpad account--dunno why they're separate), did some Page Forms reading, and have started on the data schema. For now, I'm guessing the following schema [now being maintained ​here] should do what Dr Anand wants, and this will give me something to prototype. Besides the computed-property question, other followup is to contact Dr Anand (possibly et al) about what queries he wants supported. [Note

1. contact subsequently made via email to Anand (CC Neteler and Willmes) 12 Nov 2016.
2. queries being maintained ​here

]

[tomroche]

Regarding above subtask=​update OSGeo wikihost: I have made ticket=​wiki configuration downlevel WRT security releases.

[christian]

Hello Tom,

it's great to see you are stepping forward on this. I would like to support your project, but I am afraid you use slightly different technology, that what is already in place in the wiki. It would be very good to integrate your proposal with the existing setup.

I will gladly help in implementig the data model in existing technology, i.e. SMW + Semantic Forms + mobo [3] data modelling, see as example the OSGeo Members data model [1][2].

If you want to pursue your proposal, its fine for me, but please take care of exsisting implementations, i.e. the OSGeo Member model, to not break any functionality. And make sure, that you can actually run your proposed extensions alongside the existing extension without breaking their functionality.

Thanks for consideration.

Regards, Christian

[1] ​https://github.com/cwillmes/OSGeoWiki

[2] ​https://wiki.osgeo.org/wiki/Template:OSGeo_Member

[3] ​https://github.com/Fannon/mobo

[tomroche]

summary:

1. Regarding platform/wikihost for this ​Page Forms (PF) application:
   1. as noted above, development through at least alpha-testing can be done on a freehost (i.e., a free, external SMW provider, such as ​Referata or the ​SMW sandbox), preventing problems with current wiki apps on current wikihost.
   2. deployment of all OSGeo assets should only be onto secure hosts, and moreover onto hosts for which OSGeo can maintain security (i.e., has required resources). If time/labor resources prevent appropriate maintenance of current wikihost, we should redeploy to an external provider. (See some details regarding those on the security ticket:1819; more to come.)
2. Regarding deployment automation framework (DAF):
   1. ​Mobo and ​Page Schemas (PS) seem approximately (ICBW) functionally equivalent.
   2. Mobo is apparently (ICBW) not currently available on any external SMW provider, while PS is available on at least 2 (Referata and ​WikiHoster).
   3. Mobo is less well maintained than PS, and seems much less likely to track future changes in PF (the presentation layer which both DAFs target), and is not even secure WRT Node.js (its development framework).

details:

Greetings, Christian:

Firstly, regarding your concern that

make sure, that you can actually run your proposed extensions alongside the existing extension without breaking their functionality

: note that ​my proposal has always been to do initial development through alpha-testing on a freehost, aka an external SMW provider, precisely to avoid any such collision.

This brings us to our first main (indeed, fundamental) difference. You want to use the current wikihost. That was my intention, until I learned that the current wiki is so very downlevel (see ticket:1819) as to be insecure. For confirmation that this is not merely my opinion, see ​this thread on SMW-users. Regarding contributors to that thread, note that Jeroen De Dauw is ​SMW's co-core developer and Karsten Hoffmeyer (kghbin) is the ​editor in chief of SMW's website and doc (as well as the operator of WikiHoster). So while I am certainly an SMW newbie, my concerns regarding the insecurity of the current wikihost's configuration are shared by folks who are much more experienced with SMW and MW than I and probably you.

Operating an internet-facing site as insecure as the current wikihost is, IMHO, not an option. Unfortunately, as both you and I have previously expressed, neither of us currently has spare cycles to commit to admin-ing the wiki longterm. Accordingly, unless someone steps forward and credibly commits both to update the current wikihost to a secure configuration and to maintain its security longterm, I will recommend moving the wiki instance to an external SMW provider. There are several, including some free options, and at least one (WikiHoster) that offers support in both German and English. (I'll post more on pricing and availability separately.) These providers take care to keep both the underlying platform and the MW/SMW extensions uplevel--the current wikihost fails both.

This brings us to the second main question: DAF to use. There are at least 2 options publicly available: Mobo and PS. AFAICS (and ICBW), these 2 DAFs have approximately-equal function sets, with relatively minor implementation differences. (E.g., Mobo uses YAML while PS uses XML. I prefer YAML, but I can read and write either.) As a result, I suspect (ICBW, and am learning more about both Mobo and PS) that both your model and mine can be expressed using either DAF, since both deploy to Page Forms (PF, the uplevel version of Semantic Forms, per the provider of both, Yaron Koren).

The major implementation difference between Mobo and PS is that the former is external to MW, while PS is an MW extension. This leads to the major availability difference: at least 2 external providers (Referata and WikiHoster) support PS, but I currently know of none that support Mobo. Hence I am using PS to develop the OSRC app.

There is also a social difference between Mobo and PS, regarding their developers/maintainers, which should not be ignored. Mobo is essentially ​Simon Heimler's master's thesis project: only one other person has ​committed to Mobo, and that person made exactly 1 commit. That thesis was submitted 18 Jul 2015. Of the ​commits to the Mobo repo (archived ​here), the last functional commit was 25 Jan 2016, and the last commit of any kind was ​18 Mar 2016 (archived ​here), in which Heimler withdraws his prior commitment to a version 2. Mobo is not currently even up-to-date WRT Node.js dependencies, where it already has ​security vulnerabilities. Furthermore Heimler's last post to an SMW maillist was 3 Nov 2015 (per ​this search). Given Heimler's withdrawal from the community and the code, ISTM reasonable to assume that Mobo will be unmaintained for the forseeable future.

By contrast, both PF and PS are maintained by primarily maintained by Yaron Koren, a former ​core SMW contributor who remains very active on the SMW lists. PS has a ​much larger commit community (including current and former core SMW contributors), and ​much more frequent commits. Hence it seems quite unlikely that PS will become abandonware; particularly it seems likely that PS will track any future changes in PF (unlike Mobo).

[christian]

Hello Tom,

good to hear from you. I indeed read all your querys on the SMW list, and I was lets say irritated. As you know I answered privately off-list on one of your first queries, where you were blaming a "Current Maintainer" of the wiki to not care for his responsibilites. You did not answer me until today. I decided to take no offense, but also don't care about your request as a result.

I publicly (in this very ticket) offered you to implement this or help and support you implementing this using SMW+Mobo. If you want to use different technology, just go for it. I still offer to help with mobo and SMW, and I would appreciate, If we could manage the Schemas applied to the OSGeo wiki in a collaborative code repository. If you go for an external host, do your thing, no problem.

To your technical question: Mobo is a client side application, it implements all changes on the wiki via the mediawiki API. So, its quite independent from the current Wiki version. The schema is formulated in JSON and/or YAML, actually its based on JSON-Schema definition, and can be collaboratively edited like any code.

If having the current version of the wiki is that important to you (apparently), you might want to ask yourself to step forward and doing the work to help keeping the wiki updated, instead of trying to educate others?