[OSGeoJapan-discuss] Integration of Japanese GIS formats into mainline GDAL ?
Ko Nagase
nagase ＠ georepublic.co.jp
2015年 8月 18日 (火) 18:41:51 PDT
Hi Hiroo-san,
CC: Akagi-san, Even, list
Thanks for your comments.
2015-08-19 9:58 GMT+09:00 Hiroo Imaki <hiroo ＠ angeli.org>:
> Hi Even, Akagi-san, and Ko-san
>
> Ko-san, I am sorry I missed your message you set to the list last month. I
> was on vacation and did not check email for a while.
>
> I stepped in this conversation last May and has been talking to those people
> who worked on GDAL and OGR drivers for the Japanese data. But I realized
> that I am not that useful on this topic (sorry!).
Okay, no problem.
> Here is what I found so far. For the old drivers (JPGIS driver for vector
> data and DEM raster data driver), owners of those agreed on releasing them
> with the MIT license. For the JPGIS (GML) vector driver, there is one more
> step to release it but it hasn't reached the point to release it.
Okay, thanks for information.
I guess that existence(under procedure) JPGIS (GML) v4 vector driver
implementation
is different from Akagi-san's GML driver improvement, and not conflict
with each other.
So, I think that both drivers will be available in near future.
> Anyway,I totally agree on Akagi-san's comment and people who worked on this
> should make efforts for the support of recent format rather than old formats
> that are no longer distributed from GSI.
>
> I hope we can release what we already have soon too.
Okay.
At QGIS Hackfest Tokyo/Hokkaido/Osaka (2015-08-01),
Kayama-san renamed Github "qgis-jp" group name to "OSGeo-jp".
So, we will be able to work Japanese GDAL adjustment/improvement together
after forking "OSGeo/gdal" to "OSGeo-jp/gdal".
(I am thinking to prepare JPGIS (GML) v4 dummy data, then put those to
the forked repository.)
> Akagi-san and Ko-san, great job!!
Thank you, too!
Regards,
Ko
> On Tue, Aug 18, 2015 at 4:57 PM, Even Rouault <even.rouault ＠ spatialys.com>
> wrote:
>>
>> Hi Minoru,
>> >
>> > We had an online hackathon about this topic on the same day as QGIS
>> > HackFests in Japan (2015-08-01). The purpose was to read FGD (GML)
>> > vector data (v4) with the OGR GML driver.
>> >
>> > I prepared .gfs schema files, then registered them to the GML registry
>> > and made a few changes to the GML driver code. Finally, I managed to
>> > read FGD (GML) vector data with the driver! See the work branch [1]
>> > and the hackpad [2][3] for the details.
>>
>> This is cool you managed to use the GML registry mechanism.
>>
>> I think however that the changes done in ogrgmldatasource.cpp in this
>> branch
>> should be amended to restore the previous behaviour when the namespace
>> prefix
>> is defined in the registry, and the behaviour of the branch when it is not
>> defined, so that full control is still available.
>>
>> I'd add to the todo list to add a few words to drv_gml.html to mention
>> that
>> FGD GML v4 is supported in the "Particular GML application schemas"
>> section.
>>
>> >
>> > I think we should make efforts for the support of recent format rather
>> > than old formats that are no longer distributed from GSI.
>>
>> That makes sense. Sometimes supporting old formats can be of interest when
>> people are interested in historic data that isn't available to more modern
>> formats, but this is admitedly a particular use case.
>>
>> Best regards,
>>
>> Even
>>
>> >
>> > Any comments?
>> >
>> > [1] https://github.com/minorua/gdal/commits/fgdgml_vector_1.11
>> > The branch is based on 1.11 (the current GDAL version of OSGeo4W).
>> > [2] https://hackpad.com/JPGISGML-HackFest-2015-XgBUI6qfbGP (In
>> > Japanese)
>> > [3] https://hackpad.com/JPGIS-GML-Hackathon-2015-iI1ButdfjwX (English
>> > summary)
>> >
>> > Minoru
>> >
>> > 2015-07-20 23:23 GMT+09:00 Even Rouault <even.rouault ＠ spatialys.com>:
>> > > Ko-san,
>> > >
>> > > Thanks a lot for bringing some light on this.
>> > >
>> > > I've created https://trac.osgeo.org/gdal/ticket/6046 to capture all
>> > > findings up to now and trace tasks that I've in mind that should be
>> > > completed for integretation into upstream GDAL.
>> > >
>> > > Best regards,
>> > >
>> > > Even
>> > >
>> > >> Hi Even,
>> > >> CC: Hiroo-san, Yamate-san, list
>> > >>
>> > >> Excuse me for cutting in this topic.
>> > >>
>> > >> I am neither the developer nor the player, but I tried mext_gdal
>> > >> test version (gdal110_test.zip) at last weekend and probably I
>> > >> understood
>> > >> some data related part, so I mention some of your question about
>> > >> datas.
>> > >> (But, if someone find my mistake, please point it out here...)
>> > >>
>> > >> 2015-05-01 18:59 GMT+09:00 Even Rouault <even.rouault ＠ spatialys.com>:
>> > >> > I tried the FGD ogr driver on the file
>> > >> >
>> > >> > https://github.com/msherry/PyXB-1.1.4/blob/master/examples/unicode_jp/d
>> > >> > at
>> > >> > a/shift_jis/FG- GML-13-RailCL25000-20080331-0001.xml
>> > >> > (found by googling FG-GML)
>> > >> >
>> > >> > It is not recognized by the driver. Looking more closely, the
>> > >> > driver
>> > >> > seems to expect a <GI> element that isn't found in that file. It
>> > >> > appears
>> > >> > that there are several versions of FGD GML, right ? The above file
>> > >> > is
>> > >> > apparently v3 (since there's a v3 schema next to it). Perhaps the
>> > >> > driver
>> > >> > is v4 (or a previous version) ? I tried to remove the test for the
>> > >> > presence of <GI> but it crashes later during parsing of dates,
>> > >> > apparently not liking the <gml:timePosition> element within
>> > >> > <lfSpanFr>
>> > >>
>> > >> There are 2 format types (DLD / GML), 2 recent versions(v3 / v4)
>> > >> and 2 (or more) data types (5/10m mesh DEM / General vector data
>> > >> .etc)
>> > >> in FGD data,
>> > >> and current mext_gdal test version seems to support the following
>> > >> pair.
>> > >>
>> > >> =====
>> > >> 1. FGD driver: DLD v3 (5/10m mesh DEM)
>> > >> 2. FG-GML driver: GML v3 (5/10m mesh DEM)
>> > >> 3. FGD driver: DLD v3 (General vector data)
>> > >> =====
>> > >>
>> > >> Above link which you found use GML v3, but it is not 5/10m mesh DEM,
>> > >> so you couldn't convert it.
>> > >>
>> > >> > I found this project :
>> > >> > http://mizutuu.github.io/jpgisgmlv4togml-converter/ /
>> > >> > https://github.com/mizutuu/jpgisgmlv4togml-converter
>> > >> > It includes the schema for FG-GML v4 and I can't see any reference
>> > >> > to
>> > >> > the
>> > >> > <GI> element in it either.
>> > >> > With some trickery related to encoding conversion, I managed to use
>> > >> > that
>> > >> > project to produce a standard GML file that the OGR GML driver
>> > >> > handles
>> > >> > well :
>> > >> >
>> > >> > cat FG-GML-13-RailCL25000-20080331-0001.xml | iconv
>> > >> > --from-code=shift-jis
>> > >> > -- to-code=utf-8 | sed "s/Shift_JIS/UTF-8/" | python
>> > >> > jpgisgml2gml.py >
>> > >> > out.gml
>> > >>
>> > >> And above GML v4 is not supported (both of 5/10m mesh DEM and
>> > >> General vector data) by current mext_gdal for now.
>> > >>
>> > >> > I've also found this site http://nlftp.mlit.go.jp/ksj-e/ and
>> > >> > downloaded
>> > >> > a
>> > >> > few datasets. Those I tried are not handled by the OGR FGD driver.
>> > >> > Some
>> > >> > of them look like GML with a particular application schema. They
>> > >> > are
>> > >> > unfortunately not standard enough for the OGR GML driver to
>> > >> > understand
>> > >> > them out of the box, although I believe it could be tweaked to
>> > >> > parse
>> > >> > them. Their header look like:
>> > >> >
>> > >> > <ksj:Dataset gml:id="A09Dataset"
>> > >> > xmlns:ksj="http://nlftp.mlit.go.jp/ksj/schemas/ksj-app"
>> > >> > xmlns:gml="http://www.opengis.net/gml/3.2"
>> > >> > xmlns:xlink="http://www.w3.org/1999/xlink" xm
>> > >> > lns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>> > >> > xsi:schemaLocation="http://nlftp.mlit.go.jp/ksj/schemas/ksj-app
>> > >> > KsjAppSchema- A09-v3_1.xsd">
>> > >> >
>> > >> > There's a shapefile next to them, with apparently same content.
>> > >>
>> > >> And about above MLIT NLNI(National Land Numerical Information) data,
>> > >> it is not FGD data...
>> > >>
>> > >> > So it appears there's a variety of GML-like formats ;-)
>> > >>
>> > >> Yes, sure...
>> > >>
>> > >> If you want to use mext_gdal test data, I just uploaded example dummy
>> > >> data
>> > >> to the following repository. So, please try to use those files.
>> > >> https://github.com/sanak/dev-notes/tree/master/datas/fgd
>> > >> (The dummy data location is near by Japan Wakayama prefecture,
>> > >> but on the pacific ocean.)
>> > >>
>> > >> Regards,
>> > >>
>> > >> >> Hi Even,
>> > >> >>
>> > >> >> I talk to Yamate-san who has been working on those three drivers.
>> > >> >> He
>> > >> >> explained that each driver has little bit different copy right
>> > >> >> background he need to resolve before he can merge them to the
>> > >> >> main.
>> > >> >>
>> > >> >> The first one, FGD driver, which he wrote for a project and he
>> > >> >> does
>> > >> >> not
>> > >> >> have control on its copy right. So, he would like to leave it at
>> > >> >> this
>> > >> >> moment.
>> > >> >>
>> > >> >> For the second one, FG-GML driver, he wrote it and he owns its
>> > >> >> copy
>> > >> >> right at this moment. But he said he can release with the licence
>> > >> >> you
>> > >> >> suggested.
>> > >> >>
>> > >> >> The last one, FGD ogr driver, which he also wrote and it is highly
>> > >> >> possible he can release it under the MIT licence. But he need to
>> > >> >> make
>> > >> >> sure his client can agree on this. So, he said he will contact to
>> > >> >> his
>> > >> >> client.
>> > >> >>
>> > >> >> So, the last two drivers can be merge into the main GDAL driver.
>> > >> >> It is
>> > >> >> little bit confusing background but I am sure we can sort these
>> > >> >> out.
>> > >> >>
>> > >> >> I will let you know when he is ready to work with you.
>> > >> >>
>> > >> >> Thanks!
>> > >> >>
>> > >> >> Hiroo
>> > >> >>
>> > >> >> On Thu, Apr 30, 2015 at 2:26 PM, Hiroo Imaki <hiroo ＠ angeli.org>
>> > >> >> wrote:
>> > >> >> > Hi Even,
>> > >> >> >
>> > >> >> > I am not the developer but he and I working at a same company.
>> > >> >> > If he
>> > >> >> > can agree on the license and release his efforts, that would be
>> > >> >> > wonderful for all Japanese data uses.
>> > >> >> >
>> > >> >> > He hesitates writing his message in English. So, I can be a
>> > >> >> > translator
>> > >> >> > between you guys. I hope no lost in translation!
>> > >> >> >
>> > >> >> > I will get back to you once I hear from him.
>> > >> >> >
>> > >> >> > Hiroo
>> > >> >> >
>> > >> >> > On Thu, Apr 30, 2015 at 10:45 AM, Even Rouault
>> > >> >> > <even.rouault ＠ spatialys.com
>> > >> >> >
>> > >> >> > > wrote:
>> > >> >> >> Hi,
>> > >> >> >>
>> > >> >> >> Sorry for writing in English, but I don't unfortunately speak
>> > >> >> >> Japanese.
>> > >> >> >>
>> > >> >> >> I'm one of the GDAL developers.
>> > >> >> >>
>> > >> >> >> I've noticed that there are "un-official" versions of GDAL
>> > >> >> >> floating
>> > >> >> >> around, with
>> > >> >> >> a few drivers for new Japanese-specific formats. I was
>> > >> >> >> wondering if
>> > >> >> >> there was a
>> > >> >> >> reason to keep them outside of the official GDAL tree ?
>> > >> >> >> Integrating
>> > >> >> >> them into
>> > >> >> >> mainline GDAL would probably make life of Japanese users
>> > >> >> >> easier.
>> > >> >> >> GDAL
>> > >> >> >> has already support for a lof of national specific formats,
>> > >> >> >> mainly
>> > >> >> >> in
>> > >> >> >> Europe and
>> > >> >> >> North America. Would be cool for Asia to join in ;-)
>> > >> >> >>
>> > >> >> >> In a recent discussion thread, the following link is
>> > >> >> >> mentionned:
>> > >> >> >> https://www.dropbox.com/s/4oyhncknavdk2if/gdal110_test.zip
>> > >> >> >>
>> > >> >> >> I can see that it contains 2 new GDAL drivers ( FGD: Japanese
>> > >> >> >> Fundamental Geographic Data (.xml) & FG-GML : Japanese
>> > >> >> >> Fundamental
>> > >> >> >> Geographic Data GML(.xml) ) and one OGR driver (FGD).
>> > >> >> >> Unfortunatelly there's no author or copyright information in
>> > >> >> >> the
>> > >> >> >> header of the
>> > >> >> >> corresponding source files. Would their author(s) be willing to
>> > >> >> >> release them
>> > >> >> >> under the X/MIT license and contribute them to the mainline
>> > >> >> >> GDAL
>> > >> >> >> project ? Perhaps some test files to write minimalistic
>> > >> >> >> regressions
>> > >> >> >> tests would be useful.
>> > >> >> >>
>> > >> >> >> Let me know if I can be useful for anything. I can also help
>> > >> >> >> for
>> > >> >> >> writing new
>> > >> >> >> drivers if needed.
>> > >> >> >>
>> > >> >> >> Best regards,
>> > >> >> >>
>> > >> >> >> Even
>> > >> >> >>
>> > >> >> >> --
>> > >> >> >> Spatialys - Geospatial professional services
>> > >> >> >> http://www.spatialys.com
>> > >> >> >> _______________________________________________
>> > >> >> >> OSGeoJapan-discuss mailing list
>> > >> >> >> OSGeoJapan-discuss ＠ lists.osgeo.org
>> > >> >> >> http://lists.osgeo.org/mailman/listinfo/osgeojapan-discuss
>> > >> >> >
>> > >> >> > --
>> > >> >> > Hiroo Imaki
>> > >> >> > Pacific Spatial Solutions, LLC
>> > >> >> > 1523 Chatham Colony Ct.
>> > >> >> > Reston, VA 20190
>> > >> >> > hiroo.imaki ＠ pacificspatial.com
>> > >> >> > http://www.pacificspatial.com
>> > >> >> > http://www.geopacific.org (GIS info site)
>> > >> >> > hiroo ＠ angeli.org (private)
>> > >> >
>> > >> > --
>> > >> > Spatialys - Geospatial professional services
>> > >> > http://www.spatialys.com
>> > >> > _______________________________________________
>> > >> > OSGeoJapan-discuss mailing list
>> > >> > OSGeoJapan-discuss ＠ lists.osgeo.org
>> > >> > http://lists.osgeo.org/mailman/listinfo/osgeojapan-discuss
>> > >
>> > > --
>> > > Spatialys - Geospatial professional services
>> > > http://www.spatialys.com
>> > > _______________________________________________
>> > > OSGeoJapan-discuss mailing list
>> > > OSGeoJapan-discuss ＠ lists.osgeo.org
>> > > http://lists.osgeo.org/mailman/listinfo/osgeojapan-discuss
>>
>> --
>> Spatialys - Geospatial professional services
>> http://www.spatialys.com
>> _______________________________________________
>> OSGeoJapan-discuss mailing list
>> OSGeoJapan-discuss ＠ lists.osgeo.org
>> http://lists.osgeo.org/mailman/listinfo/osgeojapan-discuss
>
>
>
>
> --
> Hiroo Imaki
> Pacific Spatial Solutions, LLC
> 1523 Chatham Colony Ct.
> Reston, VA 20190
> hiroo.imaki ＠ pacificspatial.com
> http://www.pacificspatial.com
> http://www.geopacific.org (GIS info site)
> hiroo ＠ angeli.org (private)
>
>
> _______________________________________________
> OSGeoJapan-discuss mailing list
> OSGeoJapan-discuss ＠ lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/osgeojapan-discuss
--
Ko Nagase (長瀬 興)
Georepublic Japan
mail: nagase ＠ georepublic.co.jp
web: http://georepublic.co.jp
OSGeoJapan-discuss メーリングリストの案内