[gdal-dev] Misc. subjects : OSGeo Vienna code sprint, release plans, GDAL 2.0

[Kyle Shannon]

On Tue, Apr 1, 2014 at 12:36 PM, Etienne Tourigny
<etourigny.dev at gmail.com> wrote:
> The 2 objections I have with json are :
>
> - it is so verbose that editing by hand is not as easy as .csv
> - the xml tags make file size much larger than .csv files, unless they would
> be stored in a compressed file (gzip)
>
> On the other hand, who messes with theses files on a regular basis anyway?
>
> It seems like a nice idea. In what ways would it be better than Even's
> suggestion to use sqlite?
>
>
> On Mon, Mar 31, 2014 at 4:37 PM, Dmitriy Baryshnikov <bishop.dev at gmail.com>
> wrote:
>>
>> Hi,
>>
>> I think the JSON format is good for metadata storing and representation.
>> JSON support to store string, digits, bool, dates, etc. Write such data and
>> read such data from files.
>> But need good library or set of classes to work with it in memory
>> representation.
>> For example I like  wxJSON - it support to get value by name or iterate
>> values by index.
>>
>> int majorVer = root["wxWidgets"]["Version"]["Major"].AsInt();
>>
>>
>> root["wxWidgets"]["Authors"][0] = "J. Smart";
>> root["wxWidgets"]["Authors"][1] = "V. Zeitling";
>> root["wxWidgets"]["Authors"][2] = "R. Roebling";
>>
>>
>> It consist only 3 files (json reader, json writer and json value).
>>
>> It seems to me that json-c library is more complicated. Unfortunately
>> wxJSON cannot be used in GDAL as it have bindings to wxWidgets library, but
>> the approach is interesting. By the way wxWidgets have more democratic
>> license instead of qt.
>>
>> Best regards,
>>     Dmitry
>>
>> 31.03.2014 23:03, Even Rouault пишет:
>>
>> Hi Etienne,
>>
>> Thanks for your ideas.
>>
>> Hi all,
>>
>> I have a few suggestions for gdal 2.0, based on my personal experience in
>> learning to use, enhance and maintain gdal/ogr code.
>>
>> - replace cpl/csl/string/xml code with a mainstream, modern cross-platform
>> toolkit such as QT, boost, etc.
>>
>> QT is certainly a dependency we wouldn't want to draw. Too big for some
>> embededded usage, and it would make GDAL to be practially bound by the
>> LGPL.
>> I guess standard C++ libraries classes, or perhaps boost, should do the
>> job
>> for what you mention below.
>>
>> While cpl/csl classes are robust and "do the job", they are not well
>> documented and not very intuitive for a new gdal coder. This is from my
>> personal experience, some may not agree.
>> They are also not used outside gdal, as such do not benefit from
>> enhancements as other toolkits.
>>
>> Well, at least, MapServer uses a few CPL functions : CPL minixml,
>> CPLMalloc/CPLFree, CPLReadDir, CPLFormFilename, CPLGetFilename,
>> CSLInsertString, etc..
>>
>> The drawback is that some data/methods would not be easily available in c
>> and other bindings. Also it might not be available for all platforms.
>> Existing code would still be able to use cpl/csl code but be deprecated
>> until a given release.
>>
>>
>> - upgrade/migrate gdal support file (files in .csv format such as gcs.csv)
>> reading to use containers (e.g. hash maps) instead of reading the relevant
>> .csv files every time
>>
>> Current reading of gdal support files is sub-optimal as any query for
>> support data requires reading the relevant support file(s). It would be
>> more efficient to read these once and cache into a container (such as hash
>> map). This could be done using a cross-platform toolkit mentioned earlier.
>>
>> Another option would be to use a .sqlite database with proper indices.
>> SQLite
>> would become a runtime requirement, but I don't think that would be a big
>> problem.
>>
>> - modify metadata treatment to be able to store/retrieve data in formats
>> other than strings (e.g. floats, doubles, boolean), and query the actual
>> type of a given metadata item.
>>
>> Currently there is no way to know if a given metadata item represents an
>> integer, float, double (or string).
>>
>> In the netcdf driver, I overcame this limitation in 2 ways:
>> 1) parsing the value to test if it is an int, float or double (potentially
>> error-prone)
>> 2) add an extra metadata item to specify the type of the data represented
>> (cumbersome and non-standard)
>>
>> My suggestion would be to be able to store metadata value along with its
>> data type (default string as previously). This would require some kind of
>> container for internal representation (e.g. in qt: QMap<QString,
>> QVariant>)
>> and a way to represent this on file (as .aux.xml or otherwise in files
>> that
>> support metadata) such as gtiff. Values could be stored as strings to
>> avoid
>> big/little endian conversion problems. Data type could be stored in a
>> (hidden) metadata domain, one item for each "real" metadata key.
>>
>> Interesting idea.
>>
>> Well, we need coders now ;-)
>>
>> Even
>>
>>
>>
>> _______________________________________________
>> gdal-dev mailing list
>> gdal-dev at lists.osgeo.org
>> http://lists.osgeo.org/mailman/listinfo/gdal-dev
>
>
>
> _______________________________________________
> gdal-dev mailing list
> gdal-dev at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/gdal-dev

What problem are we trying to solve exactly?  Is CPL* thought to be
buggy because it isn't exposed to the wild?  As far as the csv and
support files, is it a performance issue we are trying to solve?.  Is
the metadata system insufficient for some purposes?  It may be in
Etienne's case, due to typing, but are there other examples?  I'm just
curious.  Thanks.

-- 
Kyle