debbie failures on postgis

[robe]

I think debbie's failures in raster, crashing might be due to some upstream postgresql changes recently done to all the branches.

I reverted to tagged 10_5 and the dev run passed.

Here 2.5 is also failing similarly crashing so need to fix before we release 2.5 to see if its something we need to change or need to report upstream.

  /var/lib/jenkins/workspace/postgis/branches/3.0/regress/00-regress-install/share/contrib/postgis/rtpostgis_proc_set_search_path.sql
PostgreSQL 10.5 on x86_64-pc-linux-gnu, compiled by gcc (Debian 8.2.0-5) 8.2.0, 64-bit
  Postgis 3.0.0dev - r16729 - 2018-09-11 06:20:39
  scripts 3.0.0dev r16729
  raster scripts 3.0.0dev r16729
  GEOS: 3.7.0rc1-CAPI-1.11.0 49f66217
  PROJ: Rel. 5.1.0, June 1st, 2018
  GDAL: GDAL 2.3.1, released 2018/06/22

Running tests

 check_gdal .. ok 
 load_outdb ... ok 
 check_raster_columns .. ok 
 check_raster_overviews .. ok 
 rt_io .. ok 
 rt_bytea .. ok 
 rt_wkb .. ok 
 box3d .. ok 
 rt_addband .. ok 
 rt_band .. failed (diff expected obtained: /var/lib/jenkins/workspace/postgis/tmp/3_0_pg10w64/test_10_diff)
-----------------------------------------------------------------------------
--- rt_band_expected	2018-09-11 06:20:16.310293357 +0000
+++ /var/lib/jenkins/workspace/postgis/tmp/3_0_pg10w64/test_10_out	2018-09-11 06:25:52.143617183 +0000
@@ -12,18 +12,7 @@
 9876.54321
 987.654321
 1234.5678
-1234.5678
-1234.5678
-1234.5678
-4
-3
-2
-1
-1
-1
-NOTICE:  Invalid band index (must use 1-based). Returning original raster
-3
-NOTICE:  Invalid band index (must use 1-based). Returning original raster

-6|(1,{{7}})

+psql: FATAL:  the database system is in recovery mode
-----------------------------------------------------------------------------
 rt_valuepercent .. failed (diff expected obtained: /var/lib/jenkins/workspace/postgis/tmp/3_0_pg10w64/test_37_diff)
-----------------------------------------------------------------------------
--- rt_valuepercent_expected	2018-09-11 06:20:16.906295708 +0000
+++ /var/lib/jenkins/workspace/postgis/tmp/3_0_pg10w64/test_37_out	2018-09-11 06:25:52.435618333 +0000
@@ -1,35 +1 @@
--10.000|0.500
-3.142|0.500
--10.000|0.500
-3.142|0.500
--10.000|0.010
-0.000|0.980
-3.142|0.010
--10.000|0.500
-3.100|0.500
-3.100|0.500
--10.000|0.500
--10.000|0.500
--10.000|0.500
-3.000|0.000
-0.500
-0.010
-0.500
-0.500
-0.500
-0.500
-BEGIN
--10.000|0.010
-0.000|0.980
-3.142|0.010
--10.000|0.500
-1.000|0.000
--10.000|0.500
-3.100|0.500
--1.000|0.000
-3.100|0.500
-0.500
-0.000
-0.500
-0.000
-COMMIT
+psql: FATAL:  the database system is in recovery mode
-----------------------------------------------------------------------------
 rt_bandmetadata .. failed (diff expected obtained: /var/lib/jenkins/workspace/postgis/tmp/3_0_pg10w64/test_38_diff)

[robe]

seems to have been caused by this. So has nothing to do with raster, but use of regex functions.

​https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=f6f61d937bfddbe2a5f6a37bc26a0587117d7837

author	Andrew Gierth <rhodiumtoad@postgresql.org>	
	Tue, 28 Aug 2018 04:52:25 -0400 (09:52 +0100)
committer	Andrew Gierth <rhodiumtoad@postgresql.org>	
	Tue, 28 Aug 2018 06:55:18 -0400 (11:55 +0100)
commit	f6f61d937bfddbe2a5f6a37bc26a0587117d7837
tree	cec65e55e1c5f85a32c2421b8c17dc72327e9a0d	tree | snapshot
parent	0f3dd76f527deb81ee5ba60048df04c598c93960	commit | diff
Avoid quadratic slowdown in regexp match/split functions.

regexp_matches, regexp_split_to_table and regexp_split_to_array all
work by compiling a list of match positions as character offsets (NOT
byte positions) in the source string.

Formerly, they then used text_substr to extract the matched text; but
in a multi-byte encoding, that counts the characters in the string,
and the characters needed to reach the starting byte position, on
every call. Accordingly, the performance degraded as the product of
the input string length and the number of match positions, such that
splitting a string of a few hundred kbytes could take many minutes.

Repair by keeping the wide-character copy of the input string
available (only in the case where encoding_max_length is not 1) after
performing the match operation, and extracting substrings from that
instead. This reduces the complexity to being linear in the number of
result bytes, discounting the actual regexp match itself (which is not
affected by this patch).

In passing, remove cleanup using retail pfree() which was obsoleted by
commit ff428cded (Feb 2008) which made cleanup of SRF multi-call
contexts automatic. Also increase (to ~134 million) the maximum number
of matches and provide an error message when it is reached.

Backpatch all the way because this has been wrong forever.

Analysis and patch by me; review by Kaiting Chen.

Discussion: https://postgr.es/m/87pnyn55qh.fsf@news-spur.riddles.org.uk

see also https://postgr.es/m/87lg996g4r.fsf@news-spur.riddles.org.uk

This was committed to 9.3-12 stable branches.

[robe]

Fixed upstream. (9.3, 9.4, 9.5, 9.6, 10, 11, 12) ​https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=f7d0343ead29cd3e44eccd1c8b7cc291b79209fe

    Repair bug in regexp split performance improvements. (details)

Commit f7d0343ead29cd3e44eccd1c8b7cc291b79209fe by rhodiumtoad

Repair bug in regexp split performance improvements.
Commit c8ea87e4b introduced a temporary conversion buffer for substrings
extracted during regexp splits. Unfortunately the code that sized it was
failing to ignore the effects of ignored degenerate regexp matches, so
for regexp_split_* calls it could under-size the buffer in such cases.
Fix, and add some regression test cases (though those will only catch 
the bug if run in a multibyte encoding).
Backpatch to 9.3 as the faulty code was.
Thanks to the PostGIS project, Regina Obe and Paul Ramsey for the report
(via IRC) and assistance in analysis. Patch by me.

The file was modified	src/test/regress/expected/strings.out
The file was modified	src/test/regress/sql/strings.sql
The file was modified	src/backend/utils/adt/regexp.c