[postgis-devel] [wktraster] Core tests failure for r5841

[Jorge Arévalo]

On Thu, Jan 13, 2011 at 5:41 PM, Pierre Racine
<Pierre.Racine at sbf.ulaval.ca> wrote:
> So would everybody be happy if we just clamp the provided double value to the desired pixel type MAX or MIN when there is an overflow (e.g. for 16BSI -20000- is first clamped to -32768)?
>
> So the code would go from:
>
>            case PT_16BSI:
>            {
>                int16_t *ptr = mem;
>
>                for (i = 0; i < numval; i++)
>                    ptr[i] = (int16_t) initialvalue;
>                checkvalint = ptr[0];
>                break;
>            }
>
> to
>
>            case PT_16BSI:
>            {
>                int16_t *ptr = mem;
>
>                if (initialvalue < SHRT_MIN)
>                    initialvalue = SHRT_MIN;
>                if (initialvalue > SHRT_MAX)
>                    initialvalue = SHRT_MAX;
>
>                for (i = 0; i < numval; i++)
>                    ptr[i] = (int16_t) initialvalue;
>                checkvalint = ptr[0];
>                break;
>            }
>
> for all int types. Is this "portable, robust and numerically stable". If not, why? Because those constants are different from one architecture to the other?
>
> If the value do not overflow, do we also have to explicitly truncate double before converting to int? If we don't, should we still expect undefined results?
>
>            case PT_16BSI:
>            {
>                int16_t *ptr = mem;
>
>                initialvalue = trunc(initialvalue);
>
>                if (initialvalue < SHRT_MIN)
>                    initialvalue = SHRT_MIN;
>                if (initialvalue > SHRT_MAX)
>                    initialvalue = SHRT_MAX;
>
>                for (i = 0; i < numval; i++)
>                    ptr[i] = (int16_t) initialvalue;
>                checkvalint = ptr[0];
>                break;
>            }
>
> Pierre
>

But SHRT_MIN and SHRT_MAX can be real values for pixels. What about
storing NODATA in those cases, like Mateusz suggested?


-- 
Jorge Arévalo
Internet & Mobilty Division, DEIMOS
jorge.arevalo at deimos-space.com
http://mobility.grupodeimos.com/
http://gis4free.wordpress.com
http://geohash.org/ezjqgrgzz0g


>>-----Original Message-----
>>From: strk [mailto:strk at keybit.net]
>>Sent: 13 janvier 2011 08:48
>>To: PostGIS Development Discussion
>>Cc: Pierre Racine
>>Subject: Re: [postgis-devel] [wktraster] Core tests failure for r5841
>>
>>On Thu, Jan 13, 2011 at 01:33:02PM +0000, Mateusz Loskot wrote:
>>
>>> Once the truncation rules are defined for integral types, it should be
>>> not a problem to implement them. For instance, what choices we have to
>>> cope with -32769 value for 16BSI?
>>> - cast to int16_t and document result as undefined
>>> - truncate towards Zero by storing SHRT_MIN (SHRT_MAX for positive)
>>> - wrap around
>>> - store NODATA value
>>> - ...
>>
>>To keep things simple I'd always clamp any number < -32768 to -32768
>>and any number > 32767 to 32767.
>>
>>Makes sense from a sampling point of view too after all..
>>
>>--strk;
>>
>>  ()   Free GIS & Flash consultant/developer
>>  /\   http://strk.keybit.net/services.html
> _______________________________________________
> postgis-devel mailing list
> postgis-devel at postgis.refractions.net
> http://postgis.refractions.net/mailman/listinfo/postgis-devel
>