make install error when compiling from source while processing 'topology' subdirectory for version > 3.1.7

[louisd]

Compiling postgis on alpine/musl with postgresql 15 (also tested with 14.5) works with version 3.1.7 but other newer versions (tested with 3.2.1 and 3.3.1) fail at the make install step. Workaround found, explained below.

(./configure.sh and make were executed on both versions before the "make install" step shown here)

bash-5.1# cd /install/postgis-3.1.7/topology
bash-5.1# make install
/bin/sh /usr/local/pgsql/lib/pgxs/src/makefiles/../../config/install-sh -c -d '/usr/local/pgsql/lib'
/bin/sh /usr/local/pgsql/lib/pgxs/src/makefiles/../../config/install-sh -c -d '/usr/local/pgsql/share/contrib/postgis-3.1'
/usr/bin/install -c -m 755  postgis_topology-3.so '/usr/local/pgsql/lib/postgis_topology-3.so'
/usr/bin/install -c -m 644  topology.sql topology_upgrade.sql uninstall_topology.sql '/usr/local/pgsql/share/contrib/postgis-3.1/'


bash-5.1# cd /install/postgis-3.3.1/topology
bash-5.1# make install
/bin/sh ../config/install-sh -c -d '/usr/local/pgsql/lib'
/bin/sh: can't open '../config/install-sh': No such file or directory
make: *** [/usr/local/pgsql/lib/pgxs/src/makefiles/../../src/Makefile.shlib:471: installdirs-lib] Error 2

The difference between the two seems to be the ../config/install-sh (relative path) instead of the full path in the working version :/usr/local/pgsql/lib/pgxs/src/makefiles/../../config/install-sh)

Fix applied for now (for me) is:

ln -s /usr/local/pgsql/lib/pgxs/config /install/postgis-3.3.1

Then, execution completes correctly:

bash-5.1# ln -s /usr/local/pgsql/lib/pgxs/config /install/postgis-3.3.1
bash-5.1# make install
/bin/sh ../config/install-sh -c -d '/usr/local/pgsql/lib'
/bin/sh ../config/install-sh -c -d '/usr/local/pgsql/share/contrib/postgis-3.3'
/usr/bin/install -c -m 755  postgis_topology-3.so '/usr/local/pgsql/lib/postgis_topology-3.so'
mkdir -p /usr/local/pgsql/bin
/bin/sh ../libtool --mode=install /usr/bin/install -c loader/pgtopo_import "/usr/local/pgsql/bin/pgtopo_import"
libtool: install: /usr/bin/install -c loader/pgtopo_import /usr/local/pgsql/bin/pgtopo_import
/bin/sh ../libtool --mode=install /usr/bin/install -c loader/pgtopo_export "/usr/local/pgsql/bin/pgtopo_export"
libtool: install: /usr/bin/install -c loader/pgtopo_export /usr/local/pgsql/bin/pgtopo_export
/usr/bin/install -c -m 644  topology.sql topology_upgrade.sql uninstall_topology.sql '/usr/local/pgsql/share/contrib/postgis-3.3/'

Another workaround is to ./configure.sh --without-topology it works, but this is not an option for me since topology is needed in our database.

Also tested with the latest git clone done today.

[strk]

Does adding INSTALL=@INSTALL@ in the topology/Makefile.in and re-running configure work for you ?

I'm not sure how the INSTALL variable ends up being set differently under postgis/ and topology/ but I guess it must be due to some variable being defined or not at the time of include. Maybe srcdir := @srcdir@ which I see in postgis/Makefile.in but not in topology/Makefile.in

I cannot reproduce locally so can't easily tell if those actions fix the problem for you

[strk]

NOTE: I just found that INSTALL=@INSTALL@ is already present in the master branch for topology/Makefile.in -- what's the value you get in the generated topology/Makefile for it ? The same variable is NOT set in the 3.1 and 3.2 branches (but you do see a difference between those two ?)

[louisd]

Hi, thanks for responding!

after configure for both versions, here is what I see:

bash-5.1# more postgis-3.1.7/topology/Makefile | grep INSTALL
# NO_TEMP_INSTALL is a workaround for a 9.5dev bug. See:
NO_TEMP_INSTALL=yes
bash-5.1# more postgis-3.3.1/topology/Makefile | grep INSTALL
# NO_TEMP_INSTALL is a workaround for a 9.5dev bug. See:
NO_TEMP_INSTALL=yes
INSTALL = /usr/bin/install -c
        $(LIBTOOL) --mode=install $(INSTALL) $^ "$(DESTDIR)$(bindir)/pgtopo_import"
        $(LIBTOOL) --mode=install $(INSTALL) $^ "$(DESTDIR)$(bindir)/pgtopo_export"

Looking at Makefile.in, I have:

bash-5.1# more postgis-3.1.7/topology/Makefile.in | grep INSTALL
# NO_TEMP_INSTALL is a workaround for a 9.5dev bug. See:
NO_TEMP_INSTALL=yes
bash-5.1# more postgis-3.3.1/topology/Makefile.in | grep INSTALL
# NO_TEMP_INSTALL is a workaround for a 9.5dev bug. See:
NO_TEMP_INSTALL=yes
INSTALL = @INSTALL@
        $(LIBTOOL) --mode=install $(INSTALL) $^ "$(DESTDIR)$(bindir)/pgtopo_import"
        $(LIBTOOL) --mode=install $(INSTALL) $^ "$(DESTDIR)$(bindir)/pgtopo_export"

As for srcdir, here it is: (nothing)

bash-5.1# more postgis-3.1.7/topology/Makefile | grep srcdir
bash-5.1# more postgis-3.3.1/topology/Makefile | grep srcdir

[louisd]

If you want to reproduce the problem, here is a Dockerfile that will do the trick, just build an image based on it and it will fail at the last RUN command (make install):

FROM alpine as build

run mkdir /install
workdir /install

run apk add --no-cache wget git
run wget https://ftp.postgresql.org/pub/source/v15.0/postgresql-15.0.tar.gz
run wget https://download.osgeo.org/postgis/source/postgis-3.3.1.tar.gz

env temp_deps="alpine-sdk cmake wget git"
env musl_deps="gettext-dev libintl"
env postgres_deps="linux-headers readline-dev zlib-dev openssl-dev python3 python3-dev libxml2-dev lz4-dev zstd-dev"
env mysqlfdw_deps="mysql-dev"
env other_deps="openssl bash"

run apk add --no-cache $temp_deps $musl_deps $postgres_deps $mysqlfdw_deps $other_deps
 
run wget https://gitlab.com/rilian-la-te/musl-locales/-/archive/master/musl-locales-master.zip
run unzip musl-locales-master.zip
run cd musl-locales-master && cmake -DLOCALE_PROFILE=OFF -D CMAKE_INSTALL_PREFIX:PATH=/usr . && make -j$(nproc) && make install

# POSTGRESQL

run tar zxvf postgresql-*
#run cd postgresql-* && ./configure --with-openssl --with-python --with-libxml --with-lz4 --with-zstd && \
run cd postgresql-* && ./configure && make -j$(nproc) && make install

env PATH="/usr/local/pgsql/bin:${PATH}"

# POSTGIS
 
env postgis_deps="geos-dev gdal-dev proj-dev json-c-dev protobuf-dev protobuf-c-dev protobuf-c-compiler perl pcre-dev"
run apk add --no-cache $postgis_deps

run tar zxvf postgis-*

run cd postgis-* && ./configure
run cd postgis-* && make -j$(nproc) 

# this is where it fails
run cd postgis-* && make install

Here is the error message:

---- Making install in topology
make[1]: Entering directory '/install/postgis-3.3.1/topology'
/bin/sh ../config/install-sh -c -d '/usr/local/pgsql/lib'
/bin/sh: can't open '../config/install-sh': No such file or directory
make[1]: *** [/usr/local/pgsql/lib/pgxs/src/makefiles/../../src/Makefile.shlib:469: installdirs-lib] Error 2
make[1]: Leaving directory '/install/postgis-3.3.1/topology'
make: *** [GNUmakefile:24: install] Error 1
The command '/bin/sh -c cd postgis-* && make install' returned a non-zero code: 2
ERROR: Service 'postgis' failed to build : Build failed

You can then enter the image and look around with

docker run --rm -it (docker image sha256) bash

All files are in /install

Let me know if you want me to run further tests/check files.

Thanks!

[louisd]

A little more digging. Here is a diff of a modification I made to topologu/Makefile that fixes the problem (at least for me):

bash-5.1# diff Makefile.ori Makefile
--- Makefile.ori
+++ Makefile
@@ -71,12 +71,13 @@
 # NO_TEMP_INSTALL is a workaround for a 9.5dev bug. See:
 # http://www.postgresql.org/message-id/CAB7nPqTsR5o3g-fBi6jbsVdhfPiLFWQ_0cGU5=94Rv_8W3qvFA@mail.gmail.com
 NO_TEMP_INSTALL=yes
-include $(PGXS)

 top_builddir = ..
 VPATH := .
 INSTALL = /usr/bin/install -c
-LIBTOOL = $(SHELL) $(top_builddir)/libtool
+LIBTOOL := $(SHELL) $(top_builddir)/libtool
+
+include $(PGXS)

 # Set PERL _after_ the include of PGXS
 PERL=/usr/bin/perl

Basically move "include $(PGXS)" after the variables INSTALL and LIBTOOL are declared. And also define LIBTOOL := instead of LIBTOOL = since $(top_builddir) gets redefined in /usr/local/pgsql/lib/pgxs/src/makefiles/pgxs.mk (line 86), and using := instead of = solves this. I guess you could put LIBTOOL = $(SHELL) ../libtool instead of defining a variable and then using it.

This fixes my issue, but I wonder if it creates new problems.

[strk]

The Makefile is generated at ./configure time from Makefile.in so please modify the latter. In order to check if it creates new problems you could file a pull request on git.osgeo.org/gitea/postgis/postgis which would trigger tests run automatically with the new code. Good catch !

[strk]

Of course you can also just run make check yourself for a start :)

[louisd]

Thank you, pull request submitted, test was successful on my end.

[strk]

For the record: ​https://git.osgeo.org/gitea/postgis/postgis/pulls/111

[Sandro Santilli <strk@…>]

In ecc2bdd/git:

Fix topology install on Alpine

Move include $(PGXS) and use := for LIBTOOL
References #5264 in master branch

[strk]

I'm not sure how safe/worth it is to backport this to 3.3 and 3.2 branches. Is this a PG15 specific issue ? Alpine specific ?

[louisd]

This is Alpine specific. In fact, at first I thought it was a PG14 issue, and I waited for PG15 to try it before opening the issue, so I can confirm it's still a problem with PG14.

[Sandro Santilli <strk@…>]

In 6b2d430/git:

Fix topology install on Alpine

Move include $(PGXS) and use := for LIBTOOL
References #5264 in 3.3 branch (3.3.2dev)

[strk]

I've backported the patch to stable-3.3 branch but it doesn't apply cleanly on the stable-3.2 branch, can you check if stable-3.2 branch builds fine on alpine ? There's no LIBTOOL at all defined in there

[louisd]

Just tested with ​https://postgis.net/stuff/postgis-3.2.4dev.tar.gz

And it compiles and installs (make install) correctly. No need to fix anything.

[strk]

Thanks for testing!