PostGIS 2.4.4 rt_gdalwrap test fails with GDAL 2.3.0 & PROJ 5.1.0 on Debian unstable

[Bas Couwenberg]

As reported by Niko Tyni in ​Debian Bug #901158:

This package fails to build from source on current sid/amd64.

 ### /tmp/pgis_reg/test_50_diff ###
--- rt_gdalwarp_expected      2018-04-06 05:05:52.000000000 +0000
+++ /tmp/pgis_reg/test_50_out 2018-06-09 14:27:15.049036062 +0000
@@ -19,8 +19,8 @@
 0.17|993309|243|243|1|50.000|50.000|0.000|0.000|950760.000|1396957.000|t|t|t
 0.18|992163|10|10|1|1000.000|-1000.000|3.000|3.000|-500030.000|600000.000|t|t|t
 0.19|993310|12|12|1|1009.894|-1009.894|3.000|3.000|950691.792|1409281.783|t|t|t
-0.2|993309|12|12|1|1009.916|-1009.916|0.000|0.000|950762.305|1409088.896|t|t|t
-0.20|993309|12|12|1|1009.916|-1009.916|1.000|3.000|950742.107|1409088.896|t|t|t
+0.2|993309|12|12|1|1009.876|-1009.876|0.000|0.000|950808.447|1409091.640|t|t|t
+0.20|993309|12|12|1|1009.876|-1009.876|1.000|3.000|950788.250|1409091.640|t|t|t
 0.21|993310|24|24|1|500.000|500.000|3.000|3.000|950657.188|1397356.783|t|t|t
 0.22|993310|26|26|1|500.000|500.000|0.000|6.000|950452.000|1396632.000|t|t|t
 0.23|984269|12|8|1|0.012|-0.012|0.000|0.000|-107.029|50.206|t|t|t
@@ -63,12 +63,12 @@
 1.9|992163|11|10|1|1000.000|-1000.000|0.000|0.000|-500001.000|600000.000|t|t|t
 2.1|993310|12|12|1|1009.894|-1009.894|0.000|0.000|950732.188|1409281.783|t|t|t
 2.10|993310|24|24|1|500.000|500.000|0.000|0.000|950732.188|1397281.783|t|t|t
-2.11|993309|121|121|1|100.000|100.000|0.000|0.000|950762.305|1396988.896|t|t|t
+2.11|993309|121|121|1|100.000|100.000|0.000|0.000|950808.447|1396991.640|t|t|t
 2.12|993310|6|6|1|2000.000|2000.000|0.000|0.000|950732.188|1397281.783|t|t|t
 2.13|993310|8|8|1|1500.000|1500.000|0.000|0.000|950732.188|1397281.783|t|t|t
 2.14|993310|24|24|1|500.000|500.000|0.000|0.000|950732.188|1397281.783|t|t|t
 2.15|993310|16|16|1|750.000|750.000|0.000|0.000|950732.188|1397281.783|t|t|t
-2.2|993309|12|12|1|1009.916|-1009.916|0.000|0.000|950762.305|1409088.896|t|t|t
+2.2|993309|12|12|1|1009.876|-1009.876|0.000|0.000|950808.447|1409091.640|t|t|t
 2.3|994269|12|8|1|0.012|-0.012|0.000|0.000|-107.029|50.206|t|t|t
 2.4|||||||||||||
 2.5|993310|12|12|1|1009.894|-1009.894|0.000|0.000|950732.188|1409281.783|t|t|t
 ### end of log dumps ###
make[1]: *** [debian/rules:200: override_dh_auto_test] Error 2
make[1]: Leaving directory '/<<BUILDDIR>>/postgis-2.4.4+dfsg'
make: *** [debian/rules:111: build] Error 2

Full build log attached.

​postgis_amd64-2018-06-09T14:19:48Z.build.gz

[Bas Couwenberg]

The attached ​rt_gdalwarp.patch fixes the issue, but is likely only appropriate for systems with GDAL 2.3.0 and PROJ 5.1.0.

The underlaying issue may be related to ​pj_transform: reset error state before each call to pj_inv/pj_fwd for which Even mentioned its affect on some gdalwarp scenarios.

[Bas Couwenberg]

This patch is still applied to postgis-2.5beta1

[robe]

In 16682:

Change srid test for 993309 to be the same as 3309 in spatial_ref_sys table.
References #4104

[robe]

Bas,

I'm not setup to test GDAL 2.3.0 or Proj 5.1.0 at the moment. I noticed though that the test 993309 isn't quite what we are shipping with spatial_ref_sys. Though making it the same didn't change the answer I get.

Can you see if that helps your regress any?

If we can't come up with a workable test for both versions of proj, I suppose we can just take out that particular srid test.

[Bas Couwenberg]

Disabling rt_gdalwarp.patch and applying the changes from r16682 makes no difference:

 rt_gdalwarp .. failed (diff expected obtained: /tmp/pgis_reg/test_52_diff)
-----------------------------------------------------------------------------
--- rt_gdalwarp_expected        2018-07-03 20:26:25.000000000 +0000
+++ /tmp/pgis_reg/test_52_out   2018-08-07 11:01:22.106541719 +0000
@@ -19,8 +19,8 @@
 0.17|993309|243|243|1|50.000|50.000|0.000|0.000|950760.000|1396957.000|t|t|t
 0.18|992163|10|10|1|1000.000|-1000.000|3.000|3.000|-500030.000|600000.000|t|t|t
 0.19|993310|12|12|1|1009.894|-1009.894|3.000|3.000|950691.792|1409281.783|t|t|t
-0.2|993309|12|12|1|1009.916|-1009.916|0.000|0.000|950762.305|1409088.896|t|t|t
-0.20|993309|12|12|1|1009.916|-1009.916|1.000|3.000|950742.107|1409088.896|t|t|t
+0.2|993309|12|12|1|1009.876|-1009.876|0.000|0.000|950808.447|1409091.640|t|t|t
+0.20|993309|12|12|1|1009.876|-1009.876|1.000|3.000|950788.250|1409091.640|t|t|t
 0.21|993310|24|24|1|500.000|500.000|3.000|3.000|950657.188|1397356.783|t|t|t
 0.22|993310|26|26|1|500.000|500.000|0.000|6.000|950452.000|1396632.000|t|t|t
 0.23|984269|12|8|1|0.012|-0.012|0.000|0.000|-107.029|50.206|t|t|t
@@ -63,12 +63,12 @@
 1.9|992163|11|10|1|1000.000|-1000.000|0.000|0.000|-500001.000|600000.000|t|t|t
 2.1|993310|12|12|1|1009.894|-1009.894|0.000|0.000|950732.188|1409281.783|t|t|t
 2.10|993310|24|24|1|500.000|500.000|0.000|0.000|950732.188|1397281.783|t|t|t
-2.11|993309|121|121|1|100.000|100.000|0.000|0.000|950762.305|1396988.896|t|t|t
+2.11|993309|121|121|1|100.000|100.000|0.000|0.000|950808.447|1396991.640|t|t|t
 2.12|993310|6|6|1|2000.000|2000.000|0.000|0.000|950732.188|1397281.783|t|t|t
 2.13|993310|8|8|1|1500.000|1500.000|0.000|0.000|950732.188|1397281.783|t|t|t
 2.14|993310|24|24|1|500.000|500.000|0.000|0.000|950732.188|1397281.783|t|t|t
 2.15|993310|16|16|1|750.000|750.000|0.000|0.000|950732.188|1397281.783|t|t|t
-2.2|993309|12|12|1|1009.916|-1009.916|0.000|0.000|950762.305|1409088.896|t|t|t
+2.2|993309|12|12|1|1009.876|-1009.876|0.000|0.000|950808.447|1409091.640|t|t|t
 2.3|994269|12|8|1|0.012|-0.012|0.000|0.000|-107.029|50.206|t|t|t
 2.4|||||||||||||
 2.5|993310|12|12|1|1009.894|-1009.894|0.000|0.000|950732.188|1409281.783|t|t|t
-----------------------------------------------------------------------------

This was tested with postgis 2.5.0beta1.

[robe]

Hmm reading thru the thread. Sounds like a bug in proj 5? I assume your PROJ 5.1.0. doesn't have Even's patch in place yet right? In which case when it does this would be giving the answer we have.

I'm going to disable the 993309 srid test for now with a note to turn it back on later. I'll keep this ticket open for the time being though.

[Bas Couwenberg]

The changes Even mentioned where included as a patch in the proj (5.0.1-2) Debian package, the 5.1.0-rc1 upstream release was the first release to include the changes in proj itself.

The recent development of PROJ has changed coordinate precision quite a bit, many test cases in other projects needed to be adapted to not expect the exact same output as with PROJ 4.x. This seems to be another case like that.

Kristian Evers <kreve@…> should be able to give an authoritative answer about what changed in PROJ 5.x to cause this issue.

[robe]

In 16688:

disable tests that give different answer on Proj.5 vs. Proj.4
References #4104 for PostGIS 2.5.0

[myon]

Is this issue still relevant?

[Bas Couwenberg]

Re-enabling the test will most likely cause the test target to fail again.

[Algunenano]

Re-enabling the test works in my machine with GDAL 3 and PROJ 6.

I've created a PR to test this in travis (​https://github.com/postgis/postgis/pull/450):

• gdal 2.2 - proj 4.8
• gdal 2.2 - proj 4.9
• gdal 2.3 - proj 4.9
• gdal 2.4 - proj 5.2

[Raul Marin]

In 17616:

Re-enable gdal tests

Reverts r16688

Closes #4104
Closes ​https://github.com/postgis/postgis/pull/450

[Algunenano]

Gitlab build broke showing thw same issue. It uses PROJ 5.2

[Algunenano]

I've tried changing the definition for 993309 in several ways without any luck.

My local installation all CIs except for Gitlab (which uses Debian buster packages) work fine, and they cover multiple releases of both gdal and proj, so I don't know where this error might be coming from.

I'm going to disable again these tests and leave it be for now.

[Raul Marin]

In 17619:

Disable tests that give different answer on Proj.5 vs. Proj.4

References #4104