[gdal-dev] Grib2 Question

[Kurt Schwehr]

Why not something like this and just let me pick a small number?

#ifdef GRIB_MAX_ALLOC
const int knMaxAllloc = GRIB_MAX_ALLOC;
#else
const int knMaxAlloc = some massive number;
#endif

On Tue, Nov 7, 2017 at 3:18 PM, Even Rouault <even.rouault at spatialys.com>
wrote:

> On mardi 7 novembre 2017 14:21:27 CET Kurt Schwehr wrote:
>
> > Yeah, 1 << ## would have been better. Sigh.
>
> >
>
> > But really, someone needs to work through the logic of this stuff and do
>
> > something that actually works through what is reasonable. Code that is
>
> > intelligent and explains why it is doing is far preferable. OOM is not
>
> > okay in the world I work in, so I tried to pick a large number (basically
>
> > at random) that should let almost all cases through until someone took
> more
>
> > time.
>
>
>
> I'm afraid this is unlikely that "someone" pops up by magic to address
> that, given the time investment it would represent. For work related to
> adding a write side to the GRIB2 driver, I might try in the coming weeks to
> resync with newest version of g2clib & degrib, which by itself will be
> certainly a challenging exercice given all the patching we have done
> through the years, but adressing such issues of being robust with "hostile"
> files is definitely not in the scope of work.
>
>
>
> I'm not sure if there are other C/C++ grib1+grib2 libs that would be
> better in that aspect, while having at least the same level of
> functionality, and a permissive license.
>
>
>
> >
>
> > allowing a full range number without checking to see if the size is
>
> > reasonable is a *major* problem for me. An OOM is a fatal (as in no error
>
> > message, your process is dead) situation in my world, so allowing larger
>
> > without checking seems unpleasant.
>
> >
>
> > Perhaps a config option to allow massive files through in the short term?
>
>
>
> I think the vast majority of users are likely less concerned with
> potential OOM situations on hostile/corrupted files, but more willing to
> have a working behaviour with real world datasets, so the reverse logic
> would seem better to me.
>
>
>
> Or can we increase this threshold so that it works with datasets likes
> Roarke's one ? Kurt, what would be the maximum value you could bear ?
>
> Roarke, can you add some breakpoint / debug message to see what value of
> *ndpts you get for your dataset ?
>
>
>
> If we go with safe mode by default and a config option to disable it, then
> it should output a warning that explains which config option to enable to
> disable the limitation. + in the doc page of the driver.
>
>
>
> Even
>
>
>
> --
>
> Spatialys - Geospatial professional services
>
> http://www.spatialys.com
>



-- 
--
http://schwehr.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20171107/c10eaac5/attachment.html>