[raster] regression make check failure with ST_MapAlgebraFctNB

[robe]

I think this may just be a rounding issue. On my 9.1 windows build, its failing and diff is:

--- rt_mapalgebrafctngb_expected	Mon Nov 14 19:45:23 2011
+++ /tmp/pgis_reg_232/test_44_out	Mon Nov 14 20:02:47 2011
@@ -12,7 +12,7 @@
 t|t
 t|t
 t|t
-NOTICE:  Value set for 4BUI band got clamped from -340282346638528859811704183484516925440.000000 to 0
+NOTICE:  Value set for 4BUI band got clamped from -340282346638528860000000000000000000000.000000 to 0
 NOTICE:  Value set for 4BUI band got clamped from 18.000000 to 15
 NOTICE:  Value set for 4BUI band got clamped from 18.000000 to 15
 NOTICE:  Value set for 4BUI band got clamped from 18.000000 to 15
@@ -79,7 +79,7 @@
 NOTICE:  Value set for 4BUI band got clamped from 18.000000 to 15
 t|t
 t|t
-NOTICE:  Value set for 2BUI band got clamped from -340282346638528859811704183484516925440.000000 to 0
+NOTICE:  Value set for 2BUI band got clamped from -340282346638528860000000000000000000000.000000 to 0
 NOTICE:  Value set for 2BUI band got clamped from 9.000000 to 3
 NOTICE:  Value set for 2BUI band got clamped from 9.000000 to 3
 NOTICE:  Value set for 2BUI band got clamped from 9.000000 to 3

I'll have to double check my 9.0 and 8.4 builds.

[robe]

okay its failing on my 9.0 build with the same error, but my 8.4 is also failing but with a different error. I've attached the 8.4 one. This one looks kinda scary.

All compiled windows 7 64-bit running mingw 32-bit.

[Bborie Park]

I've fixed the problem causing the 8.4 error. I don't know what to do about the notice clamping message differences as it looks like the values are smallest possible values for that particular datatype (32BF). I've been wondering if it is time to suppress/simplify the clamping messages. Maybe simplify the clamp message so that it happens only once per raster.

[pracine]

+1

[Bborie Park]

Regina, can you try r8181? I've changed the default truncation/clamping/conversion warning behavior as it is excessive. Think 10000x10000 pixels and a warning message for every pixel! A flag was added to configure (--with-raster-dblwarning) if someone wants to enable the warnings.

[robe]

all raster regress pass now on my 9.1. I didn't look at the others yet but will assume for now they are fine too. I'll reopen if they aren't