[raster] Calling st_union() with WINDOW PARTITION crashes server

[Robins]

Brief

Calling st_union() with WINDOW PARTITION crashes the server.

Repro

WITH x AS (
  SELECT * FROM (VALUES ('A0006', 300), ('A0006', 302)) t(a,b)
)
SELECT 
 PUBLIC.st_union(NULL::raster, NULL::_unionarg) OVER (PARTITION BY a ORDER BY b)
FROM x;

SQL Output

db=> SELECT version();
-[ RECORD 1 ]---------------------------------------------------------------------------------------------------
version | PostgreSQL 13.0 on x86_64-pc-linux-gnu, compiled by gcc (GCC) 7.3.1 20180712 (Red Hat 7.3.1-6), 64-bit

db=> SELECT postgis_full_version();
-[ RECORD 1 ]--------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
postgis_full_version | POSTGIS="3.0.2 2fb2a18" [EXTENSION] PGSQL="130" GEOS="3.8.0-CAPI-1.13.1 " PROJ="Rel. 5.2.0, September 15th, 2018" GDAL="GDAL 2.4.4, released 2020/01/08" LIBXML="2.9.1" LIBJSON="0.13.1" LIBPROTOBUF="1.3.2" WAGYU="0.4.3 (Internal)" TOPOLOGY RASTER

db=> WITH x AS (
db(>   SELECT * FROM (VALUES ('A0006', 300), ('A0006', 302)) t(a,b)
db(> )
db-> SELECT
db->  PUBLIC.st_union(NULL::raster, NULL::_unionarg) OVER (PARTITION BY a ORDER BY b)
db-> FROM x;
server closed the connection unexpectedly
        This probably means the server terminated abnormally
        before or while processing the request.
The connection to the server was lost. Attempting reset: sFailed.
!?>

[robe]

This appears to be breaking at line #2437

​https://git.osgeo.org/gitea/postgis/postgis/src/branch/master/raster/rt_pg/rtpg_mapalgebra.c#L2437

if (itrset == NULL) {

Doesn't make any sense to me how that could fail, but that is what gdb is telling me is the crashing line

	itrset = palloc(sizeof(struct rt_iterator_t) * 2);
	if (itrset == NULL) {

		rtpg_union_arg_destroy(iwr);
		if (raster != NULL) {
			rt_raster_destroy(raster);
			PG_FREE_IF_COPY(pgraster, 1);
		}

		MemoryContextSwitchTo(oldcontext);
		elog(ERROR, "RASTER_union_transfn: Could not allocate memory for iterator arguments");
		PG_RETURN_NULL();
	}

[robe]

Going to push this to 3.0.4 to move out of 3.1.0 way and also since I think it is an issue in 3.0.4 too so should be fixed there as well.

[Robins]

2 other Tickets, with similar (WINDOW PARITION) crashes... may be they're related.

​https://trac.osgeo.org/postgis/ticket/4916 ​https://trac.osgeo.org/postgis/ticket/4724

[pramsey]

It's very odd, the implementations of the transfns between union(geom) and union(rast) is functionally identical, and the problem is in fact that in the raster case pgsql itself is calling the transfn in a bogus state... promising a populated aggcontext while providing an empty one. There's something very odd going on in an interaction with pgsql.

Smaller reproduction:

SELECT 
 PUBLIC.st_union(NULL::raster) OVER (ORDER BY b)
FROM (VALUES (300), (302)) t(b);

[pramsey]

The problem appears to be the finalfn freeing the aggregate state object. When I add a pfree(state) to the (working) st_union(geometry) aggregate, I can induce the same crashes. Similarly when I remove the pfree from the raster finalfn, the crashes go away.

The only concern I have about applying the fix over the various known cases is that I don't understand the mechanism behind leaving the state object intact... Magic Occurs! As far as I can tell, it's the correct magic, since the windowed results for the geometry union SEEM correct?

WITH w AS (
  SELECT 
    ST_Union(g) OVER (PARTITION BY b ORDER BY a) AS g
  FROM (VALUES ('POINT(0 0)'::geometry, 'A0006', 300), ('POINT(1 1)'::geometry, 'A0006', 302)) t(g, a, b)
)
SELECT ST_AsText(g) FROM w;

 st_astext  
------------
 POINT(0 0)
 POINT(1 1)

or with the partition wider...

WITH w AS (
  SELECT 
    ST_Union(g) OVER (PARTITION BY a ORDER BY b) AS g
  FROM (VALUES ('POINT(0 0)'::geometry, 'A0006', 300), ('POINT(1 1)'::geometry, 'A0006', 302)) t(g, a, b)
)
SELECT ST_AsText(g) FROM w;

      st_astext      
---------------------
 POINT(0 0)
 MULTIPOINT(0 0,1 1)

I'm hampered by not being 100% sure that the expected windowed results are from ST_Union(geometry) with PARTITION BY and ORDER BY are.

[pramsey]

Looking at some windows on numbers it looks like we are doing logically the same thing as the built-ins, so I guess this fix can go in.

WITH w AS (
  SELECT 
    ST_Union(g) OVER (PARTITION BY a ORDER BY b) AS g,
    Sum(b) OVER (PARTITION BY a ORDER BY b) AS s
  FROM (VALUES ('POINT(0 0)'::geometry, 'A0006', 300), ('POINT(1 1)'::geometry, 'A0006', 302)) t(g, a, b)
)
SELECT ST_AsText(g), s FROM w;

      st_astext      |  s  
---------------------+-----
 POINT(0 0)          | 300
 MULTIPOINT(0 0,1 1) | 602

[Paul Ramsey <pramsey@…>]

In ef2995b/git:

Fix crashes in aggregate functions, references #4916, #4770, #4724, #4916

[Paul Ramsey <pramsey@…>]

In 22dec99/git:

Fix crashes in aggregate functions, references #4916, #4770, #4724, #4916

[Paul Ramsey <pramsey@…>]

In 5548ad23/git:

Repair crashes in raster aggregates, references #4916, #4770, #4724, #4916