[gdal-dev] OGR ArcObjects driver

[Ragi Y. Burhum]

I changed the subject, since we were getting out of topic.

Answers inline.

On Thu, Jan 14, 2010 at 6:40 AM, Jason Roberts <jason.roberts at duke.edu>wrote:

>  Ragi,
>
>
>
> Nicely done. That is pretty funny. I'm sure there's a few others lurking on
> the list who have thought of such a thing as well. Could you please send me
> the code? Thanks!
>
>
I submitted a Trac tickets with a patch that includes the source code. Feel
free to download it from there. http://trac.osgeo.org/gdal/ticket/3332


>
>
> Do you know if it would work if invoked from Python-based geoprocessing
> tools?
>
If you are using the OGR bindings, then the answer is yes (although I have
not tried this myself, I don't foresee any problems).

We expose a lot of our work to ArcGIS as those. An interesting feature of
> these is that they can access layers created at run time by the user in the
> ArcGIS applications. They can also run both in and out of the ArcGIS
> process. The arcgisscripting Python API (and esriGeoprocessing.GPDispatch
> COM object) use named pipes to communicate with the application, or
> something like that.
>
I don't expose the internal ArcObjects to python, so you would have to use
the standard OGR python bindings. And yes, they can run inside and outside
any application.

 I am wondering if your driver would have to initialize ArcObjects in a
> special way to somehow integrate with that. I have never seen documentation
> from ESRI on how to use ArcObjects from a "script-based" geoprocessing tool
> (only from a "function-based" geoprocessing tool, which always runs in
> process and always uses ArcObjects).
>
 What is the startup cost like?
>
There are two main things that have to be done: The COM subsystem and the
ESRI licensing checkout. The startup cost in my machine is around 1s for
those two things combined.


> I have written a lot of standalone Python geoprocessing scripts that you
> run from the command line. These always take a long time to initialize the
> geoprocessor (several 5 to 20 seconds, depending on ArcGIS version) before I
> can call any APIs. I don't know if it is because the geoprocessor is
> internally allocating an ArcObjects "application" object, or if it is due to
> the license system (doesn't matter if you use a local or remote license
> server). In any case, it would be great if your driver did not suffer from
> this. I have not written any ArcObjects programs that run outside of ArcGIS
> applications, so I don't know if it would suffer this. (Sadly, I am betting
> yes…)
>
>
>
5 to 20s?!?! Wow. That sounds high!

I am only consuming the System, Geometry and Geodatabase API, so I don't
have to pay for such high costs.

> I do not know the process for submitting an OGR driver but the team seems
> very receptive to user contributions. I would be happy to review the code
> and offer comments (I'm also busy this month, so this might take some time).
> Based on some thinking I've done in the past 24 hours, I'm not sure that I
> will end up using it in my final application, but it would certainly be
> valuable. There are periodic inquiries to gdal-dev about reading file
> geodatabases, and your driver would solve that one.
>
>
>
The reason I wrote this driver was to be able to load data from FileGDB to
PostGIS directly without having to create intermediate shapefiles and such.
I have been using it like this, and it seems to be working fine.

> There are precedents for other drivers that depend on other software (e.g.
> Oracle, I think), so I don't think anyone will strenuously object to your
> driver being dependent on COM and ArcObjects. Obviously it would have to be
> an optional driver. It would be good to write some tests for it; that may be
> a requirement from the GDAL team, although I doubt they are in a position to
> run them as part of a normal GDAL release, as I don't think any of them have
> ArcGIS.
>
>
>
Yep, tests would be good. And also documentation :)

> A possible issue is that you'd probably need to recompile it every time a
> different version of ArcGIS came out. Ideally, you would maintain versions
> of it for each release of ArcGIS. The driver documentation should point out
> to the user that they must obtain the right version. Alternatively you might
> be able to craft one driver that dynamically determines the ArcGIS version
> and does the right thing.
>
>
>
Are you volunteering for some help with the documentation? That would be
nice ha ha.

Anyway, I don't suffer from being version dependent as long as ESRI does not
change the CLSIDs of the COM objects I am using (they **rarely** do this and
I cannot think of one example where they actually did change something *I*
use). Being version dependent is true for .NET components, scripting and
other languages that are tightly coupled with particular versions of dlls.
This is not the case here.

Editing, Blob and timestamp fields are not implemented yet, but reading has
been working fine for me.

I hope having FileGDB read support (even if it requires an ArcView or better
license) will help some people.

Best Regards,

Ragi Yaser Burhum

*From:* gdal-dev-bounces at lists.osgeo.org [mailto:
> gdal-dev-bounces at lists.osgeo.org] *On Behalf Of *Ragi Y. Burhum
> *Sent:* Wednesday, January 13, 2010 12:53 PM
> *To:* gdal-dev at lists.osgeo.org
>
> *Subject:* RE: [gdal-dev] Open source vector geoprocessing libraries?
>
>
>
> Date: Wed, 13 Jan 2010 10:27:43 -0500
> From: "Jason Roberts" <jason.roberts at duke.edu>
> Subject: RE: [gdal-dev] Open source vector geoprocessing libraries?
> To: "'Peter J Halls'" <P.Halls at york.ac.uk>
> Cc: 'gdal-dev' <gdal-dev at lists.osgeo.org>
> Message-ID: <008001ca9464$f4059f10$dc10dd30$@roberts at duke.edu>
> Content-Type: text/plain;       charset="US-ASCII"
>
> Peter,
>
> > are you constrained to retaining your data in an ArcGIS compatible
> format?
>
>
> We are attempting to build tools that can work with data stored in a
> variety
> of formats. Our current user community uses mostly shapefiles, ArcGIS
> personal geodatabases, and ArcGIS file geodatabases. Many of them are
> ecologists who do not have the interest or skills to deploy a real DBMS
> system. Thus we are hoping to provide tools that can work without one. This
> is one reason I was exploring how embeddable PostGIS and SpatiaLite might
> be
> in the other fork of this thread.
>
> > Until the File
> > Geodatabase format is published (later this year?) and someone has the
> effort to
> > build an OGR interface, the DBMS route is probably the best route to
> > compatibility.
>
> It would be really great for that to happen, but I'm not holding my breath.
> If it does get published, I would seriously contemplate building an OGR
> driver.
>
> I have contemplated building an ArcObjects- or arcgisscripting-based
> driver.
> This would at least allow people who have ArcGIS to use OGR to access any
> ArcGIS layer, including those created by ArcGIS's tools for joining
> arbitrary layers, etc. That would handle file geodatabases, as well as ALL
> formats accessible from ArcGIS. If such a driver existed, then we could use
> OGR as the base interface inside our application. But creating such a
> driver
> would be a lot of work and have funky dependencies because it either needs
> to use Windows COM (for ArcObjects) or Python (for arcgisscripting) to call
> the ArcGIS APIs. I am certainly capable of implementing it but because most
> of our code is in Python, it is probably easier for me to wrap OGR and
> arcgisscripting behind a common abstraction, and then have our tools work
> against that abstraction rather than OGR directly.
>
>
>
> I find it very amusing you mention this right now.
>
>
>
> Why?
>
>
>
> I asked Frank if there was an ArcObjects based OGR driver this very past
> Thursday and he said "not that I know of". What I wanted was, among other
> things, to get data out of FileGDB to PostGIS with one shot and add some
> custom behavior for a client of mine. So I spent the past three days looking
> at OGR drivers and wrote an ArcObjects based one. I got it working
> yesterday.
>
>
>
> - Right now I only instantiate 3 factories (Enterprise GDB aka ArcSDE,
> AccessDB and FileGDB). This means it reads FileGDB just fine. If you want
> more factories, the driver only has to be modified with one line to add any
> other factories and everything else would just work.
>
>
>
> - I only implemented the parts that I needed, so it is readonly (should be
> straight forward to expand if need be).
>
>
>
> - Although, it can read other GeoDatabase abstractions (Topology, Geometric
> Networks, Annotations, Cadastral Fabrics, etc), currently I am explicitly
> filtering for FeatureClasses and FeatureDatasets.
>
>
>
> - It is a ATL / COM / C++ based one, so it will only compile on Windows. It
> can be modified to use the cross platform ArcEngine SDK since all the COM
> Objects that I use are called the same and behave the same way... I just did
> not have an ArcEngine SDK installer, so I could not test this.
>
>
>
> Anyway, if you are interested in the source code, let me know. Perhaps we
> can add it as an ogr driver contribution (what is the process for that
> anyway?). I may not respond fast enough to e-mail, since the next 4 weeks
> are pretty crazy for me.
>
>
>
> - Ragi Burhum
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.osgeo.org/pipermail/gdal-dev/attachments/20100117/e1d6aad3/attachment-0001.html