Cannot build on Debian Wheezy (parser file gets rewritten)

[darkblueb]

Fails when building shp2pgsql

Background: I had a wheezy system setup in a VM, and had been using it for a number of months to test with.. but sometime recently the build no longer completed.. So thinking it might be something on my side, I built a new VM with the latest and greatest Debian, and Pg 9.1 from package.. Same error...

...

---

make[1]: Entering directory `/home/shared/srcs_wheezy/postgis_trunk/loader' /bin/bash ../libtool --mode=link gcc -g -O2 -fPIC -DPIC -Wall -Wmissing-prototypes shpopen.o dbfopen.o getopt.o shp2pgsql-core.o shpcommon.o shp2pgsql-cli.o safileio.o ../liblwgeom/liblwgeom.la -o shp2pgsql -lc libtool: link: gcc -g -O2 -fPIC -DPIC -Wall -Wmissing-prototypes shpopen.o dbfopen.o getopt.o shp2pgsql-core.o shpcommon.o shp2pgsql-cli.o safileio.o -o .libs/shp2pgsql ../liblwgeom/.libs/liblwgeom.so -lc ../liblwgeom/.libs/liblwgeom.so: undefined reference to `wkt_yylex' ../liblwgeom/.libs/liblwgeom.so: undefined reference to `wkt_lexer_init' ../liblwgeom/.libs/liblwgeom.so: undefined reference to `wkt_lexer_close' collect2: ld returned 1 exit status make[1]: * [shp2pgsql] Error 1 make[1]: Leaving directory `/home/shared/srcs_wheezy/postgis_trunk/loader' make: * [all] Error 1

---

Configure output:

---

PostGIS is now configured for i686-pc-linux-gnu

-------------- Compiler Info -------------

C compiler: gcc -g -O2 C++ compiler: g++ -g -O2

-------------- Dependencies --------------

GEOS config: /usr/local/bin/geos-config GEOS version: 3.3.2 PostgreSQL config: /usr/bin/pg_config PostgreSQL version: PostgreSQL 9.1.2 PROJ4 version: 47 Libxml2 config: /usr/bin/xml2-config Libxml2 version: 2.7.8 JSON-C support: yes PostGIS debug level: 0 Perl: /usr/bin/perl

--------------- Extensions ---------------

PostGIS Raster: disabled PostGIS Topology: enabled

-------- Documentation Generation --------

...

[strk]

Sounds like a dirty source tree, do you have a way to verify ? Fetched code from SVN ?

[pramsey]

This all traces back to *f@#*$!F(($*(# libtool and the liblwgeom.so which I so despise. Forcing the commandline utilities (r9401 ) to link statically in libtool fixes the problem in building, but hey presto, you still can't win, because the postgis.so won't load, for the same reason.

psql:/home/dbb/PRamseyBuild/postgis-svn/regress/00-regress-install/share/contrib/postgis/postgis.sql:69: ERROR:  could not load library "/home/dbb/PRamseyBuild/postgis-svn/regress/00-regress-install/lib/postgis-2.0.so": /home/dbb/PRamseyBuild/postgis-svn/regress/00-regress-install/lib/postgis-2.0.so: undefined symbol: wkt_yylex

Sure would be nice to have our internal library linked statically.

[strk]

We supposedly do have the module statically linked to liblwgeom:

postgis/regress(svn-trunk)] ldd 00-regress-install/lib/postgis-2.0.so 
        linux-vdso.so.1 =>  (0x00007fff89fff000)
        libgeos_c.so.1 => /usr/local/lib/libgeos_c.so.1 (0x00007f46e023f000)
        libproj.so.0 => /usr/local/lib/libproj.so.0 (0x00007f46dfff0000)
        libxml2.so.2 => /usr/lib/libxml2.so.2 (0x00007f46dfc9f000)
        libc.so.6 => /lib/libc.so.6 (0x00007f46df91c000)
        libgeos-3.3.2dev.so => /usr/local/lib/libgeos-3.3.2dev.so (0x00007f46df57e000)
        libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x00007f46df269000)
        libm.so.6 => /lib/libm.so.6 (0x00007f46defe6000)
        libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x00007f46dedcf000)
        libdl.so.2 => /lib/libdl.so.2 (0x00007f46debca000)
        libz.so.1 => /lib/libz.so.1 (0x00007f46de9b3000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f46e072d000)

We are not even using libtool to build that library. Run "make postgis-2.0.so" from under postgis/ dir. This is the last line I have:

gcc -g -O2 -g -Wall -O2 -fPIC -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -fno-strict-aliasing -fwrapv -g -fpic -shared  postgis_module.o lwgeom_accum.o lwgeom_spheroid.o lwgeom_ogc.o lwgeom_functions_analytic.o lwgeom_inout.o lwgeom_functions_basic.o lwgeom_btree.o lwgeom_box.o lwgeom_box3d.o lwgeom_cache.o lwgeom_geos.o lwgeom_geos_prepared.o lwgeom_geos_clean.o lwgeom_geos_relatematch.o lwgeom_export.o lwgeom_in_gml.o lwgeom_in_kml.o lwgeom_in_geojson.o lwgeom_triggers.o lwgeom_dump.o lwgeom_functions_lrs.o long_xact.o lwgeom_sqlmm.o lwgeom_rtree.o lwgeom_transform.o gserialized_typmod.o gserialized_gist_2d.o gserialized_gist_nd.o geography_inout.o geography_btree.o geography_estimate.o geography_measurement.o geometry_estimate.o  ../liblwgeom/.libs/liblwgeom.a ../libpgcommon/libpgcommon.a  -L/usr/local/lib  -lgeos_c -lproj  -lxml2    -L/usr/lib  -o postgis-2.0.so

With no trace of libtool invokation.

[strk]

Alright, I confirm it is the parser file being regenerated. Newly cloned tree, ./autogen.sh && ./configure && make && svn status:

../liblwgeom/.libs/liblwgeom.so: undefined reference to `wkt_yylex'
../liblwgeom/.libs/liblwgeom.so: undefined reference to `wkt_lexer_init'
../liblwgeom/.libs/liblwgeom.so: undefined reference to `wkt_lexer_close'
collect2: ld returned 1 exit status
make[1]: *** [shp2pgsql] Error 1
make: *** [all] Error 1
dbb@wheezy-vm:~/strk/src/postgis/trunk$ svn status
?       ERR
M       liblwgeom/lwin_wkt_lex.c

After an "svn revert -R ." and a second "make":

make[1]: Leaving directory `/home/dbb/strk/src/postgis/trunk/utils'
PostGIS was built successfully. Ready to install.

Also "make check" works fine.

This bug focus should be on finding _why_ the parser files get rewritten. And _when_ !!

[strk]

It is "make" under liblwgeom:

dbb@wheezy-vm:~/strk/src/postgis/trunk/liblwgeom$ svn status
dbb@wheezy-vm:~/strk/src/postgis/trunk/liblwgeom$ make > make.out 2> make.err
dbb@wheezy-vm:~/strk/src/postgis/trunk/liblwgeom$ svn status
?       make.out
?       make.err
M       lwin_wkt_lex.c
bb@wheezy-vm:~/strk/src/postgis/trunk/liblwgeom$ grep lwin_wkt_lex.c make.*
make.out::  -t lwin_wkt_lex.l > lwin_wkt_lex.c

Last line is interesting: ":" is the value of LEX in the Makefile. I can't find no rule explicitly requesting construction of lwin_wkt_lex though, so maybe it's a builtin feature of make ?

[strk]

Yes, it's a builtin !!! Like with .o targets, using CC, .c targets in presence of .l use LEX. Try this:

cd liblwgeom
touch test.l
make test.c

It'll use $(LEX) -t test.l > test.c

In our case $(LEX) will evaluate to ":" thus producing an empty test.c

[strk]

Actually fixing it is left as an exercise to the reader ;) .. seriously, I'm off for today ..

[pramsey]

OK, confirmed, on a clean check-out the .c file is regenerated, and, if I carefully substitute it with the svn version and build, I can get a working build and make check works.

[pramsey]

Kinda fixed at r9411. At least, we don't over-write the generated lexer file anymore. The parser doesn't seem to have the same problem (no default rule perhaps).