PostGIS 3.0 is completely broken

[robe]

I see none of the bots are happy with the latest change.

r16929

Split core and sfcgal tests allowing to run one w/out the other

Closes #4200 (detail)
by Sandro Santilli

[robe]

For reference this is what debbie is showing:

/usr/bin/perl -pi.bak -e 's,\$libdir,/var/lib/jenkins/workspace/postgis/branches/3.0/regress/00-regress-install/lib,g' /var/lib/jenkins/workspace/postgis/branches/3.0/regress/00-regress-install/share/contrib/postgis/*.sql
#make -C ../loader REGRESS=1 DESTDIR=/var/lib/jenkins/workspace/postgis/branches/3.0/regress/00-regress-install install
make -C core check
make[2]: Entering directory '/var/lib/jenkins/workspace/postgis/branches/3.0/regress/core'
/usr/bin/perl ../run_test.pl  ../loader/Point ../loader/PointM ../loader/PointZ ../loader/MultiPoint ../loader/MultiPointM ../loader/MultiPointZ ../loader/Arc ../loader/ArcM ../loader/ArcZ ../loader/Polygon ../loader/PolygonM ../loader/PolygonZ ../loader/TSTPolygon ../loader/TSIPolygon ../loader/TSTIPolygon ../loader/PointWithSchema ../loader/NoTransPoint ../loader/NotReallyMultiPoint ../loader/MultiToSinglePoint ../loader/ReprojectPts ../loader/ReprojectPtsGeog ../loader/Latin1 ../loader/Latin1-implicit ../loader/mfile dumper/literalsrid dumper/realtable affine bestsrid binary boundary chaikin filterm cluster concave_hull concave_hull_hard ctors curvetoline dump dumppoints empty estimatedextent forcecurve geography geometric_median in_geohash in_gml in_kml in_encodedpolyline iscollection legacy long_xact lwgeom_regress measures minimum_bounding_circle normalize operators orientation out_geometry out_geography polygonize polyhedralsurface postgis_type_name quantize_coordinates regress regress_bdpoly regress_index regress_index_nulls regress_management regress_selectivity regress_lrs regress_ogc regress_ogc_cover regress_ogc_prep regress_proj relate remove_repeated_points removepoint reverse setpoint simplify simplifyvw size snaptogrid split sql-mm-serialize sql-mm-circularstring sql-mm-compoundcurve sql-mm-curvepoly sql-mm-general sql-mm-multicurve sql-mm-multisurface swapordinates summary temporal tickets twkb typmod wkb wkt wmsservers knn_recheck temporal_knn hausdorff regress_buffer_params frechet offsetcurve relatematch isvaliddetail sharedpaths snap node unaryunion clean relate_bnr delaunaytriangles clipbybox2d subdivide voronoi minimum_clearance oriented_envelope in_geojson regress_brin_index regress_brin_index_3d regress_brin_index_geography mvt geobuf mvt_jsonb
Unable to find /var/lib/jenkins/workspace/postgis/branches/3.0/regress/../loader/shp2pgsql executable.
PATH is /var/lib/jenkins/workspace/pg/rel/pg10w64/bin:/var/lib/jenkins/workspace/pg/rel/pg10w64/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
Checking for shp2pgsql ... failed
make[2]: *** [Makefile:257: check] Error 2
make[2]: Leaving directory '/var/lib/jenkins/workspace/postgis/branches/3.0/regress/core'
make[1]: *** [Makefile:29: check-core] Error 2
make[1]: Leaving directory '/var/lib/jenkins/workspace/postgis/branches/3.0/regress'
make: *** [GNUmakefile:20: all] Error 1
Build step 'Execute shell' marked build as failure

Dronie:

make[2]: Entering directory `/drone/src/git.osgeo.org/gogs/postgis/postgis/regress/core'
/usr/bin/perl ../run_test.pl  ../loader/Point ../loader/PointM ../loader/PointZ ../loader/MultiPoint ../loader/MultiPointM ../loader/MultiPointZ ../loader/Arc ../loader/ArcM ../loader/ArcZ ../loader/Polygon ../loader/PolygonM ../loader/PolygonZ ../loader/TSTPolygon ../loader/TSIPolygon ../loader/TSTIPolygon ../loader/PointWithSchema ../loader/NoTransPoint ../loader/NotReallyMultiPoint ../loader/MultiToSinglePoint ../loader/ReprojectPts ../loader/ReprojectPtsGeog ../loader/Latin1 ../loader/Latin1-implicit ../loader/mfile dumper/literalsrid dumper/realtable affine bestsrid binary boundary chaikin filterm cluster concave_hull concave_hull_hard ctors curvetoline dump dumppoints empty estimatedextent forcecurve geography geometric_median in_geohash in_gml in_kml in_encodedpolyline iscollection legacy long_xact lwgeom_regress measures minimum_bounding_circle normalize operators orientation out_geometry out_geography polygonize polyhedralsurface postgis_type_name quantize_coordinates regress regress_bdpoly regress_index regress_index_nulls regress_management regress_selectivity regress_lrs regress_ogc regress_ogc_cover regress_ogc_prep regress_proj relate remove_repeated_points removepoint reverse setpoint simplify simplifyvw size snaptogrid split sql-mm-serialize sql-mm-circularstring sql-mm-compoundcurve sql-mm-curvepoly sql-mm-general sql-mm-multicurve sql-mm-multisurface swapordinates summary temporal tickets twkb typmod wkb wkt wmsservers knn_recheck temporal_knn hausdorff regress_buffer_params offsetcurve relatematch isvaliddetail sharedpaths snap node unaryunion clean relate_bnr delaunaytriangles clipbybox2d subdivide voronoi minimum_clearance oriented_envelope in_geojson regress_brin_index regress_brin_index_3d regress_brin_index_geography mvt geobuf mvt_jsonb
Unable to find /drone/src/git.osgeo.org/gogs/postgis/postgis/regress/../loader/shp2pgsql executable.
PATH is /usr/lib/postgresql/10/bin:/usr/lib/postgresql/10/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
Checking for shp2pgsql ... failed
make[2]: *** [check] Error 2
make[2]: Leaving directory `/drone/src/git.osgeo.org/gogs/postgis/postgis/regress/core'
make[1]: *** [check-core] Error 2
make[1]: Leaving directory `/drone/src/git.osgeo.org/gogs/postgis/postgis/regress'
make: *** [all] Error 1

Travis:

cc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard -O2 -fPIC -shared -o rtpostgis-3.so rtpostgis.o rtpg_internal.o rtpg_spatial_relationship.o rtpg_mapalgebra.o rtpg_utility.o rtpg_inout.o rtpg_wkb.o rtpg_geometry.o rtpg_raster_properties.o rtpg_band_properties.o rtpg_pixel.o rtpg_create.o rtpg_gdal.o rtpg_statistics.o -L/usr/local/pgsql/lib    -Wl,--as-needed -Wl,-rpath,'/usr/local/pgsql/lib',--enable-new-dtags  ../rt_core/librtcore.a ../../liblwgeom/.libs/liblwgeom.a ../../libpgcommon/libpgcommon.a -L/usr/local/lib -lgdal  -L/usr/local/lib  -lgeos_c -lproj  -ljson-c -lprotobuf-c -lxml2 -L/usr/local/lib -lSFCGAL  -Wl,--exclude-libs,ALL -Wl,-Bsymbolic-functions -Wl,-z,relro 
/usr/bin/install -c -m 755  rtpostgis-3.so '/src/postgis/regress/00-regress-install/lib/rtpostgis-3.so'
/usr/bin/install -c -m 644  rtpostgis.sql rtpostgis_for_extension.sql rtpostgis_upgrade_for_extension.sql rtpostgis_upgrade.sql rtpostgis_proc_set_search_path.sql uninstall_rtpostgis.sql rtpostgis_legacy.sql '/src/postgis/regress/00-regress-install/share/contrib/postgis/'
make[2]: Leaving directory '/src/postgis/raster/rt_pg'
make[1]: Leaving directory '/src/postgis/regress'
make: *** [all] Error 1
GNUmakefile:20: recipe for target 'all' failed
[logbt] saw 'make' exit with code:2 (INT)

[strk]

Sorry, should be fixed with r16930 (I inadvertently left the default make target under regress/ to be "check" instead of a do-nothing rule)

[strk]

r16936 hopefully makes _all_ bots happy (after a few more commits making happy some of them)

[robe]

They are still all broken

Now with this error:

 geography .. failed (diff expected obtained: /projects/postgis/tmp/3.0.0dev_pg10_geos3.7.0_gdal2.2.4w64/test_3_diff)
-----------------------------------------------------------------------------
--- geography_expected	2018-09-11 02:20:56.083471300 -0400
+++ /projects/postgis/tmp/3.0.0dev_pg10_geos3.7.0_gdal2.2.4w64/test_3_out	2018-10-29 11:03:40.592624400 -0400
@@ -1,3 +1,4 @@
+ERROR:  duplicate key value violates unique constraint "spatial_ref_sys_pkey"
 geog_distance_cached_1a|t
 geog_distance_cached_1b|t
 geog_distance_cached_1c|t
-----------------------------------------------------------------------------

[robe]

Slight correction that's what I see on debbie and winnie. Travis seems to be okay so maybe it's happening on one of the tests that only winnie and debbie do.

Bessies are having an issue too but I think theres may be different one.

Investigating.

[robe]

In 16968:

Fix sfcgal regress failure - Delete 4326 entry before readding. Also use integer format instead of string format.
References #4210

[strk]

Thanks for fixing this Regina ! Just a comment: a left-over entry in spatial-ref-sys sounds like a bug in a previously run test. I think it would make more sense to fix the previous test to cleanup after itself rather than (or in addition to) making tests tolerate a possibly dirty starting point.

Beside, we might want to _avoid_ using system SRIDs for tests (what if you run run_test.pl --nocreate and set PGIS_REG to an existing database?)

[komzpa]

Can we randomize test order so that we may catch these?

There is some test (I think regress/tickets?) that does 'delete from spatial_ref_sys;'. I once did that on my laptop's db without run_test and now I'm without my spatial_ref_sys entries.

[dbaston]

Why not begin a transaction at the start of each test and roll back at the end? Adding randomness seems like a recipe for more confusion.