ST_AsFlatGeoBuf crashes on tables when geometry column is not the first column

[robe]

CREATE TABLE test AS 
SELECT x, y, ST_Point(x,y,4326)::geometry(POINT,4326) AS geom 
FROM generate_series(1,10,1) AS x, 
 generate_series(10,1, -1) AS y;
 
-- works
SELECT ST_ASMVT(t)
FROM (SELECT x, geom FROM test) AS t;

-- crashes
SELECT ST_ASFlatGeobuf(t)
FROM (SELECT x, geom FROM test) AS t;

--this also crashes so not just derived
SELECT ST_ASFlatGeobuf(test)
FROM test;

bt looks like

[Thread 1952.0x71f8 exited with code 0]

Thread 1 received signal SIGSEGV, Segmentation fault.
[Switching to Thread 1952.0x4b50]
pg_detoast_datum_copy (datum=0x1) at fmgr.c:1733
1733            if (VARATT_IS_EXTENDED(datum))
(gdb) bt
#0  pg_detoast_datum_copy (datum=0x1) at fmgr.c:1733
#1  0x00000000711c7568 in flatgeobuf_agg_transfn (ctx=ctx@entry=0x5341e00)
    at flatgeobuf.c:527
#2  0x00000000711c7ed2 in pgis_asflatgeobuf_transfn (fcinfo=0x84d66c0)
    at lwgeom_out_flatgeobuf.c:74
#3  0x0000000000607869 in ExecAggPlainTransByVal (setno=<optimized out>,
    aggcontext=<optimized out>, pergroup=0x5341bb8, pertrans=<optimized out>,
    aggstate=<optimized out>) at execExprInterp.c:4315
#4  ExecInterpExpr (state=0x84d6470, econtext=0x5340468,
    isnull=<optimized out>) at execExprInterp.c:1713
#5  0x0000000000623193 in ExecEvalExprSwitchContext (isNull=0x520f0af,
    econtext=<optimized out>, state=<optimized out>)
    at ../../../src/include/executor/executor.h:339
#6  advance_aggregates (aggstate=0x5340030, aggstate=0x5340030)
    at nodeAgg.c:842
#7  agg_retrieve_direct (aggstate=0x5340030) at nodeAgg.c:2454
#8  ExecAgg (pstate=<optimized out>) at nodeAgg.c:2179
#9  0x000000000060c88a in ExecProcNode (node=0x5340030)
    at ../../../src/include/executor/executor.h:257
#10 ExecutePlan (execute_once=<optimized out>, dest=0x842f930,
    direction=<optimized out>, numberTuples=0, sendTuples=<optimized out>,
    operation=CMD_SELECT, use_parallel_mode=<optimized out>,
    planstate=0x5340030, estate=0x533fdf0) at execMain.c:1551
#11 standard_ExecutorRun (queryDesc=0x8438f70, direction=<optimized out>,
    count=0, execute_once=<optimized out>) at execMain.c:361
#12 0x00000000007c06eb in PortalRunSelect (portal=portal@entry=0x52dd0e0,
    forward=forward@entry=true, count=0, count@entry=2147483647,
    dest=dest@entry=0x842f930) at pquery.c:919
#13 0x00000000007c244c in PortalRun (portal=portal@entry=0x52dd0e0,
    count=count@entry=2147483647, isTopLevel=isTopLevel@entry=true,
    run_once=run_once@entry=true, dest=dest@entry=0x842f930,
    altdest=altdest@entry=0x842f930, qc=qc@entry=0x520f360) at pquery.c:763
#14 0x00000000007baacb in exec_simple_query (
    query_string=0x5314d30 "SELECT ST_ASFlatGeobuf(t)\nFROM (SELECT x, geom FROM test) AS t;") at postgres.c:1214
#15 0x00000000007be40c in PostgresMain (argc=argc@entry=1,
    argv=argv@entry=0x520f980, dbname=<optimized out>,
    username=0x1f8e90 "postgres") at postgres.c:4486
#16 0x00000000007202a1 in BackendRun (port=0x520f7b0, port=0x520f7b0)
    at postmaster.c:4506
#17 SubPostmasterMain (argc=argc@entry=3, argv=argv@entry=0x1f6d80)
    at postmaster.c:5019
#18 0x0000000000964b53 in main (argc=3, argv=0x1f6d80) at main.c:186

This is running with

PostgreSQL 14rc1 on x86_64-w64-mingw32, compiled by gcc.exe (x86_64-posix-seh-rev0, Built by MinGW-W64 project) 8.1.0, 64-bit

POSTGIS="3.2.0dev 3.2.0alpha1-19-gf48c4a86a" [EXTENSION] PGSQL="140" GEOS="3.10.0dev-CAPI-1.15.0" SFCGAL="1.3.10" PROJ="7.1.1" GDAL="GDAL 3.2.3, released 2021/04/27" LIBXML="2.9.9" LIBJSON="0.12" LIBPROTOBUF="1.2.1" WAGYU="0.5.0 (Internal)" TOPOLOGY RASTER

[robe]

example borrowed from #5000

DROP TABLE IF EXISTS foo;
CREATE TABLE foo AS
SELECT *
FROM (
VALUES 
 ('POINT(0 0)'::geometry, 'one', 1)
,('POINT(1 1)'::geometry, 'two', 2)
) AS v(g,b,i);

SELECT ST_ASFlatGeobuf(foo) OVER(ORDER BY i)
FROM foo;

This one gives an odd error:

ERROR:  tupdesc reference 00000000084DD450 is not owned by resource owner Portal
SQL state: XX000

[robe]

Okay I just noticed this doesn't crash:

SELECT ST_AsFlatGeobuf(t)
FROM (SELECT geom FROM test) AS t;

So I think it has something to do with non-geometry columns

[robe]

Okay I think I at least understand the issue now. If the first column isn't the geometry and it is a numeric, it crashes. If the first column is a geometry then it's fine.

So this crashes:

SELECT ST_ASFlatGeobuf(t)
FROM (SELECT 1 AS x, ST_Point(1,2) AS geom FROM test) AS t;

But this doesn't:

SELECT ST_ASFlatGeobuf(t)
FROM (SELECT ST_Point(1,2) AS geom, 1 AS x FROM test) AS t;

[chodgson]

Hey I don't have a source checkout or time to test this at the moment, but it looks to me like the problem is that the ctx->geom_index is used in flatgeobuf_agg_transfn() before it is ever initialized properly - it is initialized to 0 in flatgeobuf_agg_ctx_init() but it is set based on the actual data types of the tuple and/or the specified geom column name in encode_header() - which is called after geom_index is used in transfn() .

It looks like encode_header() should just be called at the end of the init? Instead of conditionally in the transfn/finalfn?

Hope that helps.

[Björn Harrtell]

Reproduced with regression test added with ​https://git.osgeo.org/gitea/postgis/postgis/commit/b24d7125a3b9de291bcc46d6d92721a66702f71c.

Fixed with ​https://git.osgeo.org/gitea/postgis/postgis/commit/f523c156430076330ba62ec11f65fee195a184b0 and additional verification with ​https://git.osgeo.org/gitea/postgis/postgis/commit/b24d7125a3b9de291bcc46d6d92721a66702f71c.