Opened 3 years ago

Closed 3 years ago

#4995 closed defect (fixed)

regress/loader tests are not re-entrant

Reported by: Bas Couwenberg Owned by: robe
Priority: medium Milestone: PostGIS 3.1.5
Component: utils/loader-dumper Version: 3.1.x
Keywords: Cc:

Description

Some loader tests are racy, they sometimes fail with: 

ERROR:  relation "loadedshp" already exists

armhf example (package build on buildd): 

 ./regress/loader/MultiPoint ............ failed (wkb testdiff expected obtained: /tmp/pgis_reg/test_116_diff)
-----------------------------------------------------------------------------
--- ./regress/loader/MultiPoint.select.expected	2021-09-04 05:03:43.000000000 +0000
+++ /tmp/pgis_reg/test_116_out	2021-09-17 03:06:56.245881317 +0000
@@ -1 +0,0 @@
-MULTIPOINT(0 1,9 -1,9 -1)
-----------------------------------------------------------------------------
 ./regress/loader/MultiPointM ... failed ( wkt test: running shp2pgsql output: /tmp/pgis_reg/loader.err)
-----------------------------------------------------------------------------
SET
SET
BEGIN
psql:/tmp/pgis_reg/loader.out:4: ERROR:  relation "loadedshp" already exists
-----------------------------------------------------------------------------
 ./regress/loader/MultiPointZ ... failed ( wkt test: running shp2pgsql output: /tmp/pgis_reg/loader.err)
-----------------------------------------------------------------------------
SET
SET
BEGIN
psql:/tmp/pgis_reg/loader.out:4: ERROR:  relation "loadedshp" already exists
-----------------------------------------------------------------------------
 ./regress/loader/Arc ... failed ( wkt test: running shp2pgsql output: /tmp/pgis_reg/loader.err)
-----------------------------------------------------------------------------
SET
SET
BEGIN
psql:/tmp/pgis_reg/loader.out:5: ERROR:  relation "loadedshp" already exists
-----------------------------------------------------------------------------
 ./regress/loader/ArcM ... failed ( wkt test: running shp2pgsql output: /tmp/pgis_reg/loader.err)
-----------------------------------------------------------------------------
SET
SET
BEGIN
psql:/tmp/pgis_reg/loader.out:4: ERROR:  relation "loadedshp" already exists
-----------------------------------------------------------------------------
 ./regress/loader/ArcZ ... failed ( wkt test: running shp2pgsql output: /tmp/pgis_reg/loader.err)
-----------------------------------------------------------------------------
SET
SET
BEGIN
psql:/tmp/pgis_reg/loader.out:4: ERROR:  relation "loadedshp" already exists
-----------------------------------------------------------------------------
 ./regress/loader/Polygon ... failed ( wkt test: running shp2pgsql output: /tmp/pgis_reg/loader.err)
-----------------------------------------------------------------------------
SET
SET
BEGIN
psql:/tmp/pgis_reg/loader.out:4: ERROR:  relation "loadedshp" already exists
-----------------------------------------------------------------------------
 ./regress/loader/PolygonM ... failed ( wkt test: running shp2pgsql output: /tmp/pgis_reg/loader.err)
-----------------------------------------------------------------------------
SET
SET
BEGIN
psql:/tmp/pgis_reg/loader.out:4: ERROR:  relation "loadedshp" already exists
-----------------------------------------------------------------------------
 ./regress/loader/PolygonZ ... failed ( wkt test: running shp2pgsql output: /tmp/pgis_reg/loader.err)
-----------------------------------------------------------------------------
SET
SET
BEGIN
psql:/tmp/pgis_reg/loader.out:4: ERROR:  relation "loadedshp" already exists
-----------------------------------------------------------------------------
 ./regress/loader/TSTPolygon ... failed ( wkt test: running shp2pgsql output: /tmp/pgis_reg/loader.err)
-----------------------------------------------------------------------------
SET
SET
BEGIN
psql:/tmp/pgis_reg/loader.out:4: ERROR:  relation "loadedshp" already exists
-----------------------------------------------------------------------------
 ./regress/loader/TSIPolygon ... failed ( wkt test: running shp2pgsql output: /tmp/pgis_reg/loader.err)
-----------------------------------------------------------------------------
SET
SET
BEGIN
psql:/tmp/pgis_reg/loader.out:4: ERROR:  relation "loadedshp" already exists
-----------------------------------------------------------------------------
 ./regress/loader/TSTIPolygon ... failed ( wkt test: running shp2pgsql output: /tmp/pgis_reg/loader.err)
-----------------------------------------------------------------------------
SET
SET
BEGIN
psql:/tmp/pgis_reg/loader.out:4: ERROR:  relation "loadedshp" already exists
-----------------------------------------------------------------------------

​Full buildlog

i386 example (GitLab CI Pipeline): 

 ./regress/loader/PolygonM ............ failed (wkb testdiff expected obtained: /tmp/pgis_reg/test_123_diff)
-----------------------------------------------------------------------------
--- ./regress/loader/PolygonM.select.expected	2021-09-08 15:38:20.000000000 +0000
+++ /tmp/pgis_reg/test_123_out	2021-09-08 15:50:21.797366851 +0000
@@ -1 +0,0 @@
-MULTIPOLYGONM(((0 0 1,0 10 7,10 10 5,10 0 3,0 0 1),(5 5 9,8 5 15,8 8 13,5 8 11,5 5 9)),((-1 -1 -1,-1 -10 -7,-10 -10 -5,-10 -1 -3,-1 -1 -1),(-5 -5 -9,-8 -5 -15,-8 -8 -13,-5 -8 -11,-5 -5 -9)))
-----------------------------------------------------------------------------
 ./regress/loader/PolygonZ ... failed ( wkt test: running shp2pgsql output: /tmp/pgis_reg/loader.err)
-----------------------------------------------------------------------------
SET
SET
BEGIN
psql:/tmp/pgis_reg/loader.out:4: ERROR:  relation "loadedshp" already exists
-----------------------------------------------------------------------------
 ./regress/loader/TSTPolygon ... failed ( wkt test: running shp2pgsql output: /tmp/pgis_reg/loader.err)
-----------------------------------------------------------------------------
SET
SET
BEGIN
psql:/tmp/pgis_reg/loader.out:4: ERROR:  relation "loadedshp" already exists
-----------------------------------------------------------------------------
 ./regress/loader/TSIPolygon ... failed ( wkt test: running shp2pgsql output: /tmp/pgis_reg/loader.err)
-----------------------------------------------------------------------------
SET
SET
BEGIN
psql:/tmp/pgis_reg/loader.out:4: ERROR:  relation "loadedshp" already exists
-----------------------------------------------------------------------------
 ./regress/loader/TSTIPolygon ... failed ( wkt test: running shp2pgsql output: /tmp/pgis_reg/loader.err)
-----------------------------------------------------------------------------
SET
SET
BEGIN
psql:/tmp/pgis_reg/loader.out:4: ERROR:  relation "loadedshp" already exists
-----------------------------------------------------------------------------

​Full buildlog

Retrying the build generally fixes the issue point to a race condition with the parallel builds.

Change History (4)

comment:1 by robe, 3 years ago

Resolution: → duplicate
Status: new → closed

Marking this as a duplicate of #4994

comment:2 by strk, 3 years ago

I think the message ERROR: relation "loadedshp" already exists only happens when there's a previous failure, as the run_test.pl is not dropping the table on failure, probably to allow for easier debugging.

comment:3 by strk, 3 years ago

Resolution: duplicate
Status: closed → reopened
Summary: regress/loader/MultiPoint test is racy → regress/loader tests are not re-entrant

I'm reopening this as the error signify the tests are not "self-contained" (they don't all cleanup after themselves).

comment:4 by Sandro Santilli <strk@…>, 3 years ago

Resolution: → fixed
Status: reopened → closed

In 016b911/git:

run_test.pl: avoid loader test failure breaking subsequent tests

Closes #4995

Still keeps the loadedshp table on failure if --nodrop is given

Note: See TracTickets for help on using tickets.