GDAL: Update FDO Thirdparty to latest GDAL Source

[gregboone]

This proposal recommends updating the FDO Thirdparty GDAL directory to contain the latest GDAL Source code.

[warmerdam]

I'd like to suggest that we not actually keep a copy of GDAL in /Thirdparty, and instead have the checkoutsvn scripts pull it from a "known good" tag in the GDAL SVN. Presumably for now from:

​http://svn.osgeo.org/svn/gdal/tags/gdal_1_4_0/

The downside is that this would mean we can't easily patch GDAL for FDO specific issues though I would think that would be a process fraught with dangers anyhow.

PS. is there any particular reason this is a WMS Provider component issue rather than "Thirdparty Components"? Perhaps the wms provider has a dependency on some particular gdal features?

[anonymous]

I will update the ticket to indicate that this is a Thirdparty Components issue.

As for your idea that we remove GDAL from Thirdparty, I am hesitant to do this without a larger plan in place on how we deal with Thirdparty components in general. Bob has mentioned a need to develop a common FDO/MG plan that would mitigate the need to keep third-party source code in our repository. MapGuide has similar issues with their Thirdparty components.

Your idea on having the build process pull GDAL from the GDAL SVN is a valid alternative, but I have some concerns on the impact. Such a change would shift the responsibility for obtaining a compatible set of GDAL source files compatible with our providers from the FDO development team to the FDO runtime build process. While it would introduce flexibility, it would leave the user without a default mechanism in place to get a build going quickly without having to worry about the specifics of pulling the code from a separate SVN.

I propose an alternative to your proposal...

• Update the default version of GDAL in Thirdparty to 1.4.x
• Add an $FDOGDAL variable to the build process (Similar to $FDO) that identifies where the GDAL source is located. The default value of this variable would point to the Thirdparty/GDAL location. The user has the option to change the value of this variable to point to a different copy of GDAL. This approach would be similar (but not identical) to how we handle ODBC and MYSQL SDK components.
• Add an option to the WMS and GDAL configure scripts that allow the user to specify the GDAL source code location as a --with option.

[gregboone]

Author: gregboone Date: 2007-03-04 13:09:51 -0500 (Sun, 04 Mar 2007) New Revisions: 2626-2638

This submission updates the default version of GDAL in Thirdparty to 1.4.x. The GDAL source files have been moved from Thirdparty/GDAL1.3 -> Thirdparty/gdal. The Windows and Linux build process has been updated to reflect the new GDAL source code location and content.

This submission introduces a new $FDOGDAL variable into the FDO build process (Similar to $FDO). $FDOGDAL identifies where the GDAL installed binaries and header files are located.

On Windows, the default value of $FDOGDAL would point to trunk/Thirdparty/gdal and would pull header files and libraries from…

trunk/Thirdparty/gdal/bin trunk/Thirdparty/gdal/lib trunk/Thirdparty/gdal/include

On Linux, the default value of $FDOGDAL would point to /usr/local and would pull header files and libraries from…

/usr/local/lib /usr/local/include

With the introduction of this variable, users now have the option to not build GDAL in trunk/Thirdparty/gdal and instead link against an installed version of GDAL that is different from the version included in /trunk/Thirdparty/gdal. The user can identify the location of their custom install by changing the value of the $FDOGDAL variable to point to their GDAL installation. This approach is similar (but not identical) to how we handle ODBC and MYSQL SDK components.

NOTE: By default, the trunk/build_thirdparty.sh and trunk/build_thirdparty.bat scripts build and install GDAL.

The following files were added/removed/modified

Added:

trunk/Thirdparty/gdal trunk/Thirdparty/Thirdparty_gdal.sh trunk/Thirdparty/Thirdparty_gdal.sln

Removed:

trunk/Thirdparty/GDAL1.3/

Modified:

trunk/build_linux.sh trunk/setenvironment.bat trunk/setenvironment.sh

trunk/Thirdparty/build.bat trunk/Thirdparty/Makefile.am trunk/Thirdparty/Thirdparty.sh trunk/Thirdparty/Thirdparty.sln trunk/Thirdparty/Thirdparty_wms.sh trunk/Thirdparty/Thirdparty_wms.sln

trunk/Providers/GDAL/configure trunk/Providers/GDAL/configure.in trunk/Providers/GDAL/config.status trunk/Providers/GDAL/Src/Makefile.am trunk/Providers/GDAL/Src/Provider/Makefile.am trunk/Providers/GDAL/Src/Provider/RFP.vcproj trunk/Providers/GDAL/Src/UnitTest/UnitTest.vcproj

trunk/Providers/WMS/configure trunk/Providers/WMS/configure.in trunk/Providers/WMS/Src/Makefile.am trunk/Providers/WMS/Src/Provider/Makefile.am trunk/Providers/WMS/Src/Provider/WMSProvider.vcproj trunk/Providers/WMS/Src/UnitTest/UnitTest.vcproj

[anonymous]

Greg,

I have updated to the latest fdo repository contents, and was somewhat surprised to discover that now when I run build_thirdparty.sh the scripts attempt to build and install GDAL into /usr/local.

This seems somewhat dangerous as it is likely to overwrite an existing GDAL installation.

I am wondering if there ought to be two variables. One controlling whether the Thirdparty copy of GDAL will be built and installed, and one controlling which copy of GDAL gets used when building fdogdal, the WMS provider and so forth.

In this case the default might be for setenvironment.sh to try and find a gdal-config and if found use the existing GDAL install it describes. If not found, we could build the third party components, and install them in the /usr/fdo-3.3.0 tree.

I think this could be a model for some other dependencies as well, such as xerces, and xalan.

[warmerdam]

The previous was from Frank of course.

[gregboone]

I see your point about the install and am willing to change it.

However, I am unclear what you mean by gdal-config. Can you elaborate on your proposed plan?

The gdal files can be installed into /usr/local/fdo3.3.0 using the gdal configure script and seting the prefix parameter to the fdo install directory. A side effect of installing into /usr/local/fdo-3.3.0 would be that the gdal header files would be installed into the fdo include directory. Is this a recommended approach?

[gregboone]

I am somewhat concerned about introducing another environment variable into the build process. Each one adds a little more complexity to the situation. What if we changed the current build process to key off the value of FDOGDAL? If it points to /usr/local/fdo-3-3-0 then the build process will build Thirdparty/GDAL and install in /usr/local/fdo-3-3-0. Otherwise, the Thirdparty/GDAL will not be built and the FDO providers will pick-up gdal from the previous installed location such as /user/local?
For example...

setenvironment.sh

export FDOGDAL=/usr/local/fdo-3.3.0

echo ""
echo "NOTE: The setenvironment.sh script sets the installation location for "
echo "the GDAL SDK files to /usr/local/fdo-3.3.0. If this value remains unchanged,"
echo "the FDO build process will build the version of GDAL located in Thirdpaty/gdal"
echo "and will install the resulting files in /usr/local/fdo-3.3.0. The FDO build"
echo "process will then use that location when building the GDAL and"
echo "WMS providers. If you wish to build the FDO GDAL or WMS Providers"
echo "using a previously installed version of GDAL, modify the setenvironment.sh "
echo "script and set FDOGDAL to point to the existing GDAL installation."
echo "For example: /user/local (The default GDAL installation path)."
echo ""

Thirdparty/Thirdparty.sh

## Build Thirdparty_WMS/GDAL
if test "$FDOGDAL" == "/user/local/fdo-3.3.0"; then

pushd . >& /dev/null
cd gdal
echo Building gdal
cd src
chmod a+x ./configure
echo Build GDAL library with the following settings:
echo gif support - internal
echo jpeg support - internal
echo png support - internal
echo tiff support - internal
echo geotiff support - internal
echo libz support - internal
echo python support - no
echo OGR support - no
echo postgreSQL support - no
./configure --with-gif=internal --with-jpeg=internal --with-png=internal --with-libtiff=internal --with-geotiff=internal --without-ogr --with-pg=no --with-python=no --with-libz=internal --prefix=/usr/local/fdo-3.3.0
make
make install
popd >& /dev/null

else

echo "Building Thirdparty/GDAL libraries skipped. FDOGDAL points to a previously installed version of GDAL at $FDOGDAL"

fi

[warmerdam]

Greg,

This approach seems reasonable.

gdal-config is a script which reports information about how the pre-installed GDAL is setup.

warmerda@amd64[61]% ~/bld/bin/gdal-config --help Usage: gdal-config [OPTIONS] Options:

[--prefix[=DIR]] [--libs] [--dep-libs] [--cflags] [--version] [--ogr-enabled] [--formats]

warmerda@amd64[62]% ~/bld/bin/gdal-config --prefix /home/warmerda/bld warmerda@amd64[63]% ~/bld/bin/gdal-config --libs -L/home/warmerda/bld/lib -lgdal warmerda@amd64[64]% ~/bld/bin/gdal-config --cflags -I/home/warmerda/bld/include

So an alternative approach might be to have setenvironment.sh default to trying to run gdal-config. If successful it could use the found GDAL, otherwise it could default to the thirdparty logic.

But your approach is ok too. It is just up to people to modify setenvironment.sh to point to /usr/local or something similar if they have their own install.

[gregboone]

Linking issues occur on RH EL 4. libgdal.la references invalid .libs location.

[jbirch]

Another alternative to maintaining SVN copies of dependancies, and to requiring developers to do this themselves, would be to use the SVN Externals feature:

​http://svnbook.red-bean.com/en/1.0/ch07s03.html

This would help to prevent bloat in the repository over time.

Jason