[OSGeoJapan-discuss] Integration of Japanese GIS formats into mainline GDAL ?

Even Rouault even.rouault @ spatialys.com
2015年 10月 8日 (木) 02:03:42 PDT


Hi all,

I was wondering if there had been progress on this ? I can see a 
https://github.com/OSGeo-jp/gdal repository, but it doesn't seem to contain 
changes for now.

Best regards,

Even

> Hi Hiroo-san,
> CC: Akagi-san, Even, list
> 
> Thanks for your comments.
> 
> 2015-08-19 9:58 GMT+09:00 Hiroo Imaki <hiroo at angeli.org>:
> > Hi Even, Akagi-san, and Ko-san
> > 
> > Ko-san, I am sorry I missed your message you set to the list last month.
> > I was on vacation and did not check email for a while.
> > 
> > I stepped in this conversation last May and has been talking to those
> > people who worked on GDAL and OGR drivers for the Japanese data. But I
> > realized that I am not that useful on this topic (sorry!).
> 
> Okay, no problem.
> 
> > Here is what I found so far. For the old drivers (JPGIS driver for vector
> > data and DEM raster data driver), owners of those agreed on releasing
> > them with the MIT license. For the JPGIS (GML) vector driver, there is
> > one more step to release it but it hasn't reached the point to release
> > it.
> 
> Okay, thanks for information.
> I guess that existence(under procedure) JPGIS (GML) v4 vector driver
> implementation
> is different from Akagi-san's GML driver improvement, and not conflict
> with each other.
> So, I think that both drivers will be available in near future.
> 
> > Anyway,I totally agree on Akagi-san's comment and people who worked on
> > this should make efforts for the support of recent format rather than
> > old formats that are no longer distributed from GSI.
> > 
> > I hope we can release what we already have soon too.
> 
> Okay.
> At QGIS Hackfest Tokyo/Hokkaido/Osaka (2015-08-01),
> Kayama-san renamed Github "qgis-jp" group name to "OSGeo-jp".
> So, we will be able to work Japanese GDAL adjustment/improvement together
> after forking "OSGeo/gdal" to "OSGeo-jp/gdal".
> (I am thinking to prepare JPGIS (GML) v4 dummy data, then put those to
> the forked repository.)
> 
> > Akagi-san and Ko-san, great job!!
> 
> Thank you, too!
> 
> Regards,
> 
> Ko
> 
> > On Tue, Aug 18, 2015 at 4:57 PM, Even Rouault
> > <even.rouault at spatialys.com>
> > 
> > wrote:
> >> Hi Minoru,
> >> 
> >> > We had an online hackathon about this topic on the same day as QGIS
> >> > HackFests in Japan (2015-08-01). The purpose was to read FGD (GML)
> >> > vector data (v4) with the OGR GML driver.
> >> > 
> >> > I prepared .gfs schema files, then registered them to the GML registry
> >> > and made a few changes to the GML driver code. Finally, I managed to
> >> > read FGD (GML) vector data with the driver! See the work branch [1]
> >> > and the hackpad [2][3] for the details.
> >> 
> >> This is cool you managed to use the GML registry mechanism.
> >> 
> >> I think however that the changes done in ogrgmldatasource.cpp in this
> >> branch
> >> should be amended to restore the previous behaviour when the namespace
> >> prefix
> >> is defined in the registry, and the behaviour of the branch when it is
> >> not defined, so that full control is still available.
> >> 
> >> I'd add to the todo list to add a few words to drv_gml.html to mention
> >> that
> >> FGD GML v4 is supported in the "Particular GML application schemas"
> >> section.
> >> 
> >> > I think we should make efforts for the support of recent format rather
> >> > than old formats that are no longer distributed from GSI.
> >> 
> >> That makes sense. Sometimes supporting old formats can be of interest
> >> when people are interested in historic data that isn't available to
> >> more modern formats, but this is admitedly a particular use case.
> >> 
> >> Best regards,
> >> 
> >> Even
> >> 
> >> > Any comments?
> >> > 
> >> > [1] https://github.com/minorua/gdal/commits/fgdgml_vector_1.11
> >> > 
> >> >     The branch is based on 1.11 (the current GDAL version of OSGeo4W).
> >> > 
> >> > [2] https://hackpad.com/JPGISGML-HackFest-2015-XgBUI6qfbGP  (In
> >> > Japanese)
> >> > [3] https://hackpad.com/JPGIS-GML-Hackathon-2015-iI1ButdfjwX  (English
> >> > summary)
> >> > 
> >> > Minoru
> >> > 
> >> > 2015-07-20 23:23 GMT+09:00 Even Rouault <even.rouault at spatialys.com>:
> >> > > Ko-san,
> >> > > 
> >> > > Thanks a lot for bringing some light on this.
> >> > > 
> >> > > I've created https://trac.osgeo.org/gdal/ticket/6046 to capture all
> >> > > findings up to now and trace tasks that I've in mind that should be
> >> > > completed for integretation into upstream GDAL.
> >> > > 
> >> > > Best regards,
> >> > > 
> >> > > Even
> >> > > 
> >> > >> Hi Even,
> >> > >> CC: Hiroo-san, Yamate-san, list
> >> > >> 
> >> > >> Excuse me for cutting in this topic.
> >> > >> 
> >> > >> I am neither the developer nor the player, but I tried mext_gdal
> >> > >> test version (gdal110_test.zip) at last weekend and probably I
> >> > >> understood
> >> > >> some data related part, so I mention some of your question about
> >> > >> datas.
> >> > >> (But, if someone find my mistake, please point it out here...)
> >> > >> 
> >> > >> 2015-05-01 18:59 GMT+09:00 Even Rouault 
<even.rouault at spatialys.com>:
> >> > >> > I tried the FGD ogr driver on the file
> >> > >> > 
> >> > >> > https://github.com/msherry/PyXB-1.1.4/blob/master/examples/unicod
> >> > >> > e_jp/d at
> >> > >> > a/shift_jis/FG- GML-13-RailCL25000-20080331-0001.xml
> >> > >> > (found by googling FG-GML)
> >> > >> > 
> >> > >> > It is not recognized by the driver. Looking more closely, the
> >> > >> > driver
> >> > >> > seems to expect a <GI> element that isn't found in that file. It
> >> > >> > appears
> >> > >> > that there are several versions of FGD GML, right ? The above
> >> > >> > file is
> >> > >> > apparently v3 (since there's a v3 schema next to it). Perhaps the
> >> > >> > driver
> >> > >> > is v4 (or a previous version) ? I tried to remove the test for
> >> > >> > the presence of <GI> but it crashes later during parsing of
> >> > >> > dates, apparently not liking the <gml:timePosition> element
> >> > >> > within <lfSpanFr>
> >> > >> 
> >> > >> There are 2 format types (DLD / GML), 2 recent versions(v3 / v4)
> >> > >> and 2 (or more) data types (5/10m mesh DEM / General vector data
> >> > >> .etc)
> >> > >> in FGD data,
> >> > >> and current mext_gdal test version seems to support the following
> >> > >> pair.
> >> > >> 
> >> > >> =====
> >> > >> 1. FGD driver: DLD v3 (5/10m mesh DEM)
> >> > >> 2. FG-GML driver: GML v3 (5/10m mesh DEM)
> >> > >> 3. FGD driver: DLD v3 (General vector data)
> >> > >> =====
> >> > >> 
> >> > >> Above link which you found use GML v3, but it is not 5/10m mesh
> >> > >> DEM, so you couldn't convert it.
> >> > >> 
> >> > >> > I found this project :
> >> > >> > http://mizutuu.github.io/jpgisgmlv4togml-converter/  /
> >> > >> > https://github.com/mizutuu/jpgisgmlv4togml-converter
> >> > >> > It includes the schema for FG-GML v4 and I can't see any
> >> > >> > reference to
> >> > >> > the
> >> > >> > <GI> element in it either.
> >> > >> > With some trickery related to encoding conversion, I managed to
> >> > >> > use that
> >> > >> > project to produce a standard GML file that the OGR GML driver
> >> > >> > handles
> >> > >> > well :
> >> > >> > 
> >> > >> > cat FG-GML-13-RailCL25000-20080331-0001.xml | iconv
> >> > >> > --from-code=shift-jis
> >> > >> > -- to-code=utf-8 | sed "s/Shift_JIS/UTF-8/" | python
> >> > >> > jpgisgml2gml.py >
> >> > >> > out.gml
> >> > >> 
> >> > >> And above GML v4 is not supported (both of 5/10m mesh DEM and
> >> > >> General vector data) by current mext_gdal for now.
> >> > >> 
> >> > >> > I've also found this site http://nlftp.mlit.go.jp/ksj-e/ and
> >> > >> > downloaded
> >> > >> > a
> >> > >> > few datasets. Those I tried are not handled by the OGR FGD
> >> > >> > driver. Some
> >> > >> > of them look like GML with a particular application schema. They
> >> > >> > are
> >> > >> > unfortunately not standard enough for the OGR GML driver to
> >> > >> > understand
> >> > >> > them out of the box, although I believe it could be tweaked to
> >> > >> > parse
> >> > >> > them. Their header look like:
> >> > >> > 
> >> > >> > <ksj:Dataset gml:id="A09Dataset"
> >> > >> > xmlns:ksj="http://nlftp.mlit.go.jp/ksj/schemas/ksj-app"
> >> > >> > xmlns:gml="http://www.opengis.net/gml/3.2"
> >> > >> > xmlns:xlink="http://www.w3.org/1999/xlink" xm
> >> > >> > lns:xsi="http://www.w3.org/2001/XMLSchema-instance"
> >> > >> > xsi:schemaLocation="http://nlftp.mlit.go.jp/ksj/schemas/ksj-app
> >> > >> > KsjAppSchema- A09-v3_1.xsd">
> >> > >> > 
> >> > >> > There's a shapefile next to them, with apparently same content.
> >> > >> 
> >> > >> And about above MLIT NLNI(National Land Numerical Information)
> >> > >> data, it is not FGD data...
> >> > >> 
> >> > >> > So it appears there's a variety of GML-like formats ;-)
> >> > >> 
> >> > >> Yes, sure...
> >> > >> 
> >> > >> If you want to use mext_gdal test data, I just uploaded example
> >> > >> dummy data
> >> > >> to the following repository. So, please try to use those files.
> >> > >> https://github.com/sanak/dev-notes/tree/master/datas/fgd
> >> > >> (The dummy data location is near by Japan Wakayama prefecture,
> >> > >> but on the pacific ocean.)
> >> > >> 
> >> > >> Regards,
> >> > >> 
> >> > >> >> Hi Even,
> >> > >> >> 
> >> > >> >> I talk to Yamate-san who has been working on those three
> >> > >> >> drivers. He
> >> > >> >> explained that each driver has little bit different copy right
> >> > >> >> background he need to resolve before he can merge them to the
> >> > >> >> main.
> >> > >> >> 
> >> > >> >> The first one, FGD driver, which he wrote for a project and he
> >> > >> >> does
> >> > >> >> not
> >> > >> >> have control on its copy right. So, he would like to leave it at
> >> > >> >> this
> >> > >> >> moment.
> >> > >> >> 
> >> > >> >> For the second one, FG-GML driver, he wrote it and he owns its
> >> > >> >> copy
> >> > >> >> right at this moment. But he said he can release with the
> >> > >> >> licence you
> >> > >> >> suggested.
> >> > >> >> 
> >> > >> >> The last one, FGD ogr driver, which he also wrote and it is
> >> > >> >> highly possible he can release it under the MIT licence. But he
> >> > >> >> need to make
> >> > >> >> sure his client can agree on this. So, he said he will contact
> >> > >> >> to his
> >> > >> >> client.
> >> > >> >> 
> >> > >> >> So, the last two drivers can be merge into the main GDAL driver.
> >> > >> >> It is
> >> > >> >> little bit confusing background but I am sure we can sort these
> >> > >> >> out.
> >> > >> >> 
> >> > >> >> I will let you know when he is ready to work with you.
> >> > >> >> 
> >> > >> >> Thanks!
> >> > >> >> 
> >> > >> >> Hiroo
> >> > >> >> 
> >> > >> >> On Thu, Apr 30, 2015 at 2:26 PM, Hiroo Imaki <hiroo at angeli.org>
> >> > >> >> 
> >> > >> >> wrote:
> >> > >> >> > Hi Even,
> >> > >> >> > 
> >> > >> >> > I am not the developer but he and I working at a same company.
> >> > >> >> > If he
> >> > >> >> > can agree on the license and release his efforts, that would
> >> > >> >> > be wonderful for all Japanese data uses.
> >> > >> >> > 
> >> > >> >> > He hesitates writing his message in English. So, I can be a
> >> > >> >> > translator
> >> > >> >> > between you guys. I hope no lost in translation!
> >> > >> >> > 
> >> > >> >> > I will get back to you once I hear from him.
> >> > >> >> > 
> >> > >> >> > Hiroo
> >> > >> >> > 
> >> > >> >> > On Thu, Apr 30, 2015 at 10:45 AM, Even Rouault
> >> > >> >> > <even.rouault at spatialys.com
> >> > >> >> > 
> >> > >> >> > > wrote:
> >> > >> >> >> Hi,
> >> > >> >> >> 
> >> > >> >> >> Sorry for writing in English, but I don't unfortunately speak
> >> > >> >> >> Japanese.
> >> > >> >> >> 
> >> > >> >> >> I'm one of the GDAL developers.
> >> > >> >> >> 
> >> > >> >> >> I've noticed that there are "un-official" versions of GDAL
> >> > >> >> >> floating
> >> > >> >> >> around, with
> >> > >> >> >> a few drivers for new Japanese-specific formats. I was
> >> > >> >> >> wondering if
> >> > >> >> >> there was a
> >> > >> >> >> reason to keep them outside of the official GDAL tree ?
> >> > >> >> >> Integrating
> >> > >> >> >> them into
> >> > >> >> >> mainline GDAL would probably make life of Japanese users
> >> > >> >> >> easier.
> >> > >> >> >> GDAL
> >> > >> >> >> has already support for a lof of national specific formats,
> >> > >> >> >> mainly
> >> > >> >> >> in
> >> > >> >> >> Europe and
> >> > >> >> >> North America. Would be cool for Asia to join in ;-)
> >> > >> >> >> 
> >> > >> >> >> In a recent discussion thread, the following link is
> >> > >> >> >> 
> >> > >> >> >> mentionned:
> >> > >> >> >>  https://www.dropbox.com/s/4oyhncknavdk2if/gdal110_test.zip
> >> > >> >> >> 
> >> > >> >> >> I can see that it contains 2 new GDAL drivers ( FGD: Japanese
> >> > >> >> >> Fundamental Geographic Data (.xml) & FG-GML :  Japanese
> >> > >> >> >> Fundamental
> >> > >> >> >> Geographic Data GML(.xml) ) and one OGR driver (FGD).
> >> > >> >> >> Unfortunatelly there's no author or copyright information in
> >> > >> >> >> the
> >> > >> >> >> header of the
> >> > >> >> >> corresponding source files. Would their author(s) be willing
> >> > >> >> >> to release them
> >> > >> >> >> under the X/MIT license and contribute them to the mainline
> >> > >> >> >> GDAL
> >> > >> >> >> project ? Perhaps some test files to write minimalistic
> >> > >> >> >> regressions
> >> > >> >> >> tests would be useful.
> >> > >> >> >> 
> >> > >> >> >> Let me know if I can be useful for anything. I can also help
> >> > >> >> >> for
> >> > >> >> >> writing new
> >> > >> >> >> drivers if needed.
> >> > >> >> >> 
> >> > >> >> >> Best regards,
> >> > >> >> >> 
> >> > >> >> >> Even
> >> > >> >> >> 
> >> > >> >> >> --
> >> > >> >> >> Spatialys - Geospatial professional services
> >> > >> >> >> http://www.spatialys.com
> >> > >> >> >> _______________________________________________
> >> > >> >> >> OSGeoJapan-discuss mailing list
> >> > >> >> >> OSGeoJapan-discuss at lists.osgeo.org
> >> > >> >> >> http://lists.osgeo.org/mailman/listinfo/osgeojapan-discuss
> >> > >> >> > 
> >> > >> >> > --
> >> > >> >> > Hiroo Imaki
> >> > >> >> > Pacific Spatial Solutions, LLC
> >> > >> >> > 1523 Chatham Colony Ct.
> >> > >> >> > Reston, VA  20190
> >> > >> >> > hiroo.imaki at pacificspatial.com
> >> > >> >> > http://www.pacificspatial.com
> >> > >> >> > http://www.geopacific.org (GIS info site)
> >> > >> >> > hiroo at angeli.org (private)
> >> > >> > 
> >> > >> > --
> >> > >> > Spatialys - Geospatial professional services
> >> > >> > http://www.spatialys.com
> >> > >> > _______________________________________________
> >> > >> > OSGeoJapan-discuss mailing list
> >> > >> > OSGeoJapan-discuss at lists.osgeo.org
> >> > >> > http://lists.osgeo.org/mailman/listinfo/osgeojapan-discuss
> >> > > 
> >> > > --
> >> > > Spatialys - Geospatial professional services
> >> > > http://www.spatialys.com
> >> > > _______________________________________________
> >> > > OSGeoJapan-discuss mailing list
> >> > > OSGeoJapan-discuss at lists.osgeo.org
> >> > > http://lists.osgeo.org/mailman/listinfo/osgeojapan-discuss
> >> 
> >> --
> >> Spatialys - Geospatial professional services
> >> http://www.spatialys.com
> >> _______________________________________________
> >> OSGeoJapan-discuss mailing list
> >> OSGeoJapan-discuss at lists.osgeo.org
> >> http://lists.osgeo.org/mailman/listinfo/osgeojapan-discuss
> > 
> > --
> > Hiroo Imaki
> > Pacific Spatial Solutions, LLC
> > 1523 Chatham Colony Ct.
> > Reston, VA  20190
> > hiroo.imaki at pacificspatial.com
> > http://www.pacificspatial.com
> > http://www.geopacific.org (GIS info site)
> > hiroo at angeli.org (private)
> > 
> > 
> > _______________________________________________
> > OSGeoJapan-discuss mailing list
> > OSGeoJapan-discuss at lists.osgeo.org
> > http://lists.osgeo.org/mailman/listinfo/osgeojapan-discuss

-- 
Spatialys - Geospatial professional services
http://www.spatialys.com


OSGeoJapan-discuss メーリングリストの案内