upgrade to GDAL 1.9

[jef]

are thee plans to upgrade GDAL to 1.9?

[warmerdam]

Jurgen,

I'll try to upgrade the "gdal" package to 1.9 in the next few weeks. I will likely following the existing pattern where I keep around a gdal18.dll in the package so that existing package continue to work though 1.8 plugins will likely disappear.

[pcav]

Please:

./configure --with-spatialite=yes

Thanks a lot.

[jef]

Replying to pcav:

./configure --with-spatialite=yes

configure isn't used in a vc build. And there is no spatialite package for OSGeo4W.

[warmerdam]

Ouch, somehow I had thought spatialite was packaged.

Paolo, can you confirm that you really need the spatialite library linked in? I believe GDAL/OGR still has fairly respectable spatialite data access even when just using the sqlite library.

It not already being packaged is a big wrench in the works with regard to using spatialite in OSGeo4W.

[jef]

Replying to warmerdam:

Ouch, somehow I had thought spatialite was packaged.

we have an internal spatialite in qgis and pyspatialite, but there's no common library (package).

[rouault]

Frank, linking to libspatialite is needed for sure for write support. And for benefiting from spatial index, it is likely too (in theory perhaps not, but I haven't verified recently)

[aperi2007]

Hi,

the gdal native support for spatialite is limited to the old 2.3.1 version of spatilaite. The 2.3.1 version is a very old version that support only 2D and has an obsolete spatial indexes. Using the spatialite.dll and the new driver splite added from Furieri gdal was able to support the 3D format of spatialite and the new spatialite indexes. Also it support the new spatialite system table named "history table".

[warmerdam]

jef,

Do you have any interested in factoring out spatialite as a package? Is this something Paolo or others might do?

[brushtyler]

I can do it, if you agree of course.

spatialite-amalgation has an internal-private copy of sqlite (the one in OSGeo4W is very old) and requires freexl (missing in OSGeo4W, so I would also package it).

If you think the amalgation is not a good option 'cause of the internal sqlite, we should consider to update the sqlite package (the latest sqlite is v3.7.9, OSGeo4W's one is v3.3.8). Anyway I don't know how much effort can require this task.

[warmerdam]

The ideal would be to avoid the internal amalgamation but I'm not sure what the issues are. To be honest I've always ignored spatialite because I was offended by it's practice of keeping an internal copy of sqlite.

In any event, if you package spatialite (with sqlite embedded), I'll try to package GDAL using it.

[warmerdam]

+brushtyler - see last comment.

[brushtyler]

Replying to warmerdam:

The ideal would be to avoid the internal amalgamation but I'm not sure what the issues are. To be honest I've always ignored spatialite because I was offended by it's practice of keeping an internal copy of sqlite.

Maybe the problem is the minimum sqlite version required from some spatialite features.

if you package spatialite (with sqlite embedded), I'll try to package GDAL using it.

Last month I compiled GDAL-dev against spatialite-amalgation without any problems.

I'm going to open a ticket about spatialite packaging.

[esseffe]

Replying to warmerdam:

few clarifications about SpatiaLite:

a) the obsolete v.2.3.1 released in 2009 Jul 6 is now officially *deprecated*

b) the current "stable" version is v.3.0.1 released on 2012 Jan 3

c) anyway the next v.3.1.0 is now in the Release Candidate stage, and will surely be released before Easter.

about libspatialite and libspatialite-amalgamation: *two* different flavors of the library are available:

• libspatialite is a canonical library completely depending on external dependencies (libsqlite3, libiconv, libgeos, libproj and libfreexl)
  

• libspatialite-amalgamation is exactly the same, but includes an internal private copy of the most recent SQLite.

Using the amalgamation is absolutely discouraged: this one is simply intended to be a "tricky-version" mainly supporting Windows (or any other "odd" platform), may be useful to some developers when no recent libsqlite is available at all (e.g. when attempting to build on the top as OsGeo4W as currently configured).

I strongly agree with FrankW: using the "private internal copy" of libsqlite is really dangerous and harmful, and should be carefully avoided as far as possible.

Using the "canonical" libspatialite isn't at all difficult: it simply requires as a prerequisite to build a reasonably up-to-date libsqlite. a really simple and plain task using any possible compiler on every platform AFAIK.

bye Sandro

[esseffe]

Replying to brushtyler:

Maybe the problem is the minimum sqlite version required from some spatialite features.

Few clarifications about SQLite:

a) sqlite3.dll currently distributed by OsGeo4W corresponds to v.3.3.8, released on 2006 Oct 9 [about six years ago]

b) SQLite usually releases an updated version more or less every 60 days; may be more often if some critical bug is detected.

c) SQLite current version is v.3.7.10 released on 2012 Jan 16; the next v.3.7.11 is expected to be released on April 16.

As officially stated by the SQLite developers:

"Version 3.7.10 of SQLite is recommended for all new development. Upgrading from version 3.7.6.3, 3.7.7, 3.7.7.1, 3.7.8, or 3.7.9 is optional. Upgrading from all other SQLite versions is recommended."

In other words: for sure any SQLite version released before 2011 May 19 is affected by at least one serious critical bug, now positively identified and resolved.

Enumerating the many optimizations and enhancements applied during the last six years will require a lot of space. Just to mention the most relevant ones:

• many more PRAGMA directives are now supported
  

• a lot of old APIs are now declared to be deprecated, and many new enhanced APIs has been introduced.
  

• a brand new journaling system has been implemented (many concurrent read-access are now effectively supported, avoiding deadlocks).
  

• the low-level interface to file-system has been completely refactored, and this has a strong positive impact on Windows
  

• an effective R*Tree implementation has been added
  

• now the Primary / Foreign Key constraints are effectively implemented, thus allowing to enforce strong referential integrity
  
  

Anyway, all recent versions are still able to read and write any DB-file created using any earlier version subsequent to v.3.0.0 released on 2004 June 18. And any obsolete version including v.3.3.0 released on 2006 Jan 10 is still able to read and write any DB-file created by any recent version.

So I cannot foresee any valid reason discouraging from updating SQLite on a regular basis, following as closest as possible the natural evolution of this package.

bye Sandro

[brushtyler]

Replying to esseffe:

I strongly agree with FrankW: using the "private internal copy" of libsqlite is really dangerous and harmful, and should be carefully avoided as far as possible.

Using the "canonical" libspatialite isn't at all difficult: it simply requires as a prerequisite to build a reasonably up-to-date libsqlite.

From my side, I can compile libspatialite with or without amalgation, but of course I need a newer libsqlite if we choose the second option.

Is there a good reason to keep that version of libsqlite within OSGeo4w?

[warmerdam]

There is no particular reason to keep an old sqlite in OSGeo4W but if we upgrade it we must be mindful of ABI or other changes that may affect packaging using it. It looks like the various GDAL packages, qgis-common and qgis-dev depend on sqlite3. Note that it is not very easy to rebuild some of the older GDAL packages.

[rouault]

From my experience, the sqlite developers are at least, if not more, as careful as GDAL not to break API/ABI. For example they have _v2 version of API calls when they need to add extra parameters, the structures have an integer field at their beginning to define their revision number/ABI, etc... I know some users just happily drop newer sqlite3.dll from the sqlite website instead of the current OSGeo one.

[warmerdam]

OK preliminary steps taken.

• gdal package upgraded to 1.9 branch.
• gdal-python package upgraded to 1.9 branch.
• added a gdal18dll package to provide the old 1.8 DLL for compatability.

Let me know when the sqlite package(s) are upgraded.

[brushtyler]

Replying to warmerdam:

Let me know when the sqlite package(s) are upgraded.

Looking at ​version seems the sqlite3 package has no maintainer, so I'm going to package it too.

[pcav]

Now also spatialite has been packaged, thanks Giuseppe