[raster] Possible issue with resize function outdb rasters

[robe]

I have this folder of aircraft pictures that I registered as outdb and then ran a query for resizing. They are all bmp files and for some reason some of them when I resize turn black. It doesn't seem to matter what percent or width /height I apply for resizing.

The commands I am using to register are:

raster2pgsql -F -Y -R C:/jet_photos/*.bmp staging_pics | psql

The query I was using to get resized versions looks like this:

SELECT 
  ST_AsPNG(ST_Resize(rast,0.5,0.5 )) As thumb_pic
FROM staging_pics WHERE filename = 'Grand Caravan.bmp';

A lot of them resize fine and some don't but all look okay if no resize operation is applied.

I'm using the latest winnie 64-bit 9.2 buildbot build.

PostgreSQL 9.2.2, compiled by Visual C++ build 1600, 64-bit POSTGIS="2.1.0SVN r11004" GEOS="3.4.0dev-CAPI-1.8.0 r0" PROJ="Rel. 4.8.0, 6 March 2012" GDAL="GDAL 1.9.1, released 2012/05/15" LIBXML="2.7.8" LIBJSON="UNKNOWN" RASTER

I'll send you a couple of good and bad pictures (those that resize fine and some that don't). I would attach to this ticket, but they are client pictures so can't disclose publicly.

[robe]

Just sent you the pics, but it might have something to do with outdb. I just reloaded them without the -R switch and all resize fine.

I'm going to try once more with -R to see if it behaves the same.

[robe]

Okay confirmed. The issue is only with out of db rasters. I do get a lot of errors when loading in both cases

errors like:

INFO: Using default geotransform matrix (0, 1, 0, 0, 0, -1) for raster: photos/Lear 35.bmp
ERROR: diff_rastinfo: Could not run raster alignment test

and only 1 like this (which resizes just fine)

INFO: Using default geotransform matrix (0, 1, 0, 0, 0, -1) for raster: photos/Lear 55.bmp
WARNING: Different number of bands found in the set of rasters being converted to PostGIS raster
WARNING: Different pixel types found for band 2 in the set of rasters being converted to PostGIS raster

I'm not seeing a correlation between the error notices (which seem more like warnings)since I just throw a bag of non-georeferenced pictures at the database and said register so those are expected.

[Bborie Park]

I did some quick checking of the good vs bad pictures with gdalinfo. What is very strange is the following. For your good pics, the output looks something like...

Driver: BMP/MS Windows Device Independent Bitmap
Files: GOOD FILE.bmp
Size is 200, 140
Coordinate System is `'
Corner Coordinates:
Upper Left  (    0.0,    0.0)
Lower Left  (    0.0,  140.0)
Upper Right (  200.0,    0.0)
Lower Right (  200.0,  140.0)
Center      (  100.0,   70.0)
Band 1 Block=200x1 Type=Byte, ColorInterp=Red
Band 2 Block=200x1 Type=Byte, ColorInterp=Green
Band 3 Block=200x1 Type=Byte, ColorInterp=Blue

For the bad pics...

Driver: BMP/MS Windows Device Independent Bitmap
Files: BAD FILE.bmp
Size is 269, 187
Coordinate System is `'
Origin = (-1890.000000000000000,1890.000000000000000)
Pixel Size = (3780.000000000000000,-3780.000000000000000)
Corner Coordinates:
Upper Left  (   -1890.000,    1890.000) 
Lower Left  (   -1890.000, -704970.000) 
Upper Right ( 1014930.000,    1890.000) 
Lower Right ( 1014930.000, -704970.000) 
Center      (  506520.000, -351540.000) 
Band 1 Block=269x1 Type=Byte, ColorInterp=Red
Band 2 Block=269x1 Type=Byte, ColorInterp=Green
Band 3 Block=269x1 Type=Byte, ColorInterp=Blue

So for some crazy reason, the bad pics have some sort of geotransform set. For one of the bad pictures loaded in the database, try

UPDATE foo SET rast = ST_SetGeoReference(rast, '1 0 0 -1 0 0', 'GDAL') WHERE rid = BAD_PIC_RID

And rerun the ST_Resize().

The error should be present regardless of out-db or in-db.

[robe]

You mean set on my out_db one or in_db. I set on my in db and the picture is still fine when resized. I set on my out db and outdb one is still broken.

[robe]

gdalinfo "PG:host=localhost port=5432 dbname='fl' user='postgres' password='whatever' table=staging_pics_local where='filename=\'Grand Caravan.bmp\''"

Returns the below but it resizes fine at 0.5,0.5 haven't tried others.

Driver: PostGISRaster/PostGIS Raster driver
Files: none associated
Size is 284, 177
Origin = (-1890.000000000000000,1890.000000000000000)
Pixel Size = (3780.000000000000000,-3780.000000000000000)
Corner Coordinates:
Upper Left  (   -1890.000,    1890.000)
Lower Left  (   -1890.000, -667170.000)
Upper Right ( 1071630.000,    1890.000)
Lower Right ( 1071630.000, -667170.000)
Center      (  534870.000, -332640.000)
Band 1 Block=284x177 Type=Byte, ColorInterp=Red
Band 2 Block=284x177 Type=Byte, ColorInterp=Green
Band 3 Block=284x177 Type=Byte, ColorInterp=Blue

My outdb version however which doesn't resize (gives black image) {{{gdalinfo "PG:host=localhost port=5432 dbname='fl' user='postgres' password='whatever' table=staging_pics_outdb where='filename=\'Grand Caravan.bmp\"}}

Driver: PostGISRaster/PostGIS Raster driver
Files: none associated
Size is 284, 177
Origin = (-1890.000000000000000,1890.000000000000000)
Pixel Size = (3780.000000000000000,-3780.000000000000000)
Corner Coordinates:
Upper Left  (   -1890.000,    1890.000)
Lower Left  (   -1890.000, -667170.000)
Upper Right ( 1071630.000,    1890.000)
Lower Right ( 1071630.000, -667170.000)
Center      (  534870.000, -332640.000)
Band 1 Block=284x177 Type=Byte, ColorInterp=Red
Band 2 Block=284x177 Type=Byte, ColorInterp=Green
Band 3 Block=284x177 Type=Byte, ColorInterp=Blue

They look the same to me though I would expect the outdb one to say there is a file associated with it though.

[Bborie Park]

Strange. I'll have to do some testing on the out-db code, which I need to rewrite anyways.

[Bborie Park]

I just realized what is probably causing the problem. When the offline band is loaded, it has to find where the chunk of data to fetch from the raster is. So, it needs the geotransforms from the raster in and out of the database and this is where things are going wrong. I'll need to see what is exactly happening.

[Bborie Park]

Can you try r11008? I think I fixed the problem, which affects out-db bands only as you experienced.

[robe]

Yap that fixed it. I tried without reloading the outdb pics and with reloading in outdb and both cases worked.

I was only half joking before about "I'll just use PostGIS to categorize my project photos", but it's turning out to be surprisingly useful especially with applications like flight management where I need to install PostGIS anyway.