<div>I changed the subject, since we were getting out of topic.</div><div><br></div>Answers inline.<br><br><div class="gmail_quote">On Thu, Jan 14, 2010 at 6:40 AM, Jason Roberts <span dir="ltr"><<a href="mailto:jason.roberts@duke.edu" target="_blank">jason.roberts@duke.edu</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div lang="EN-US" link="blue" vlink="purple">
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;color:#003399">Ragi,</span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;color:#003399"> </span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;color:#003399">Nicely done. That is pretty funny. I'm sure there's a few others
lurking on the list who have thought of such a thing as well. Could you please
send me the code? Thanks!</span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;color:#003399"></span></p></div></div></blockquote><div><br></div><div>I submitted a Trac tickets with a patch that includes the source code. Feel free to download it from there. <a href="http://trac.osgeo.org/gdal/ticket/3332">http://trac.osgeo.org/gdal/ticket/3332</a></div>
<div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div lang="EN-US" link="blue" vlink="purple"><div><p class="MsoNormal"><span style="font-size:10.0pt;color:#003399"> </span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;color:#003399">Do you know if it would work if invoked from Python-based
geoprocessing tools? </span></p></div></div></blockquote><div>If you are using the OGR bindings, then the answer is yes (although I have not tried this myself, I don't foresee any problems).</div><div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div lang="EN-US" link="blue" vlink="purple"><div><p class="MsoNormal"><span style="font-size:10.0pt;color:#003399">We expose a lot of our work to ArcGIS as those. An
interesting feature of these is that they can access layers created at run time
by the user in the ArcGIS applications. They can also run both in and out of
the ArcGIS process. The arcgisscripting Python API (and
esriGeoprocessing.GPDispatch COM object) use named pipes to communicate with
the application, or something like that.</span></p></div></div></blockquote><div>I don't expose the internal ArcObjects to python, so you would have to use the standard OGR python bindings. And yes, they can run inside and outside any application.</div>
<div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div lang="EN-US" link="blue" vlink="purple"><div><p class="MsoNormal"><span style="font-size:10.0pt;color:#003399"> I am wondering if your driver would
have to initialize ArcObjects in a special way to somehow integrate with that. I
have never seen documentation from ESRI on how to use ArcObjects from a "script-based"
geoprocessing tool (only from a "function-based" geoprocessing tool,
which always runs in process and always uses ArcObjects).<span style="color:rgb(0, 0, 0);font-size:small"> </span></span></p></div></div></blockquote><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div lang="EN-US" link="blue" vlink="purple"><div>
<p class="MsoNormal"><span style="font-size:10.0pt;color:#003399">What is the startup cost like? </span></p></div></div></blockquote><div>There are two main things that have to be done: The COM subsystem and the ESRI licensing checkout. The startup cost in my machine is around 1s for those two things combined.</div>
<div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div lang="EN-US" link="blue" vlink="purple"><div><p class="MsoNormal"><span style="font-size:10.0pt;color:#003399">I have written a lot of standalone
Python geoprocessing scripts that you run from the command line. These always
take a long time to initialize the geoprocessor (several 5 to 20 seconds,
depending on ArcGIS version) before I can call any APIs. I don't know if it is
because the geoprocessor is internally allocating an ArcObjects "application"
object, or if it is due to the license system (doesn't matter if you use a
local or remote license server). In any case, it would be great if your driver
did not suffer from this. I have not written any ArcObjects programs that run
outside of ArcGIS applications, so I don't know if it would suffer this.
(Sadly, I am betting yes…)</span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;color:#003399"> </span></p></div></div></blockquote><div>5 to 20s?!?! Wow. That sounds high!</div><div><br></div><div>I am only consuming the System, Geometry and Geodatabase API, so I don't have to pay for such high costs. </div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div lang="EN-US" link="blue" vlink="purple"><div>
<p class="MsoNormal"><span style="font-size:10.0pt;color:#003399">I do not know the process for submitting an OGR driver but the team
seems very receptive to user contributions. I would be happy to review the code
and offer comments (I'm also busy this month, so this might take some time).
Based on some thinking I've done in the past 24 hours, I'm not sure that I will
end up using it in my final application, but it would certainly be valuable. There
are periodic inquiries to gdal-dev about reading file geodatabases, and your
driver would solve that one.</span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;color:#003399"> </span></p></div></div></blockquote><div>The reason I wrote this driver was to be able to load data from FileGDB to PostGIS directly without having to create intermediate shapefiles and such. I have been using it like this, and it seems to be working fine.</div>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div lang="EN-US" link="blue" vlink="purple"><div>
<p class="MsoNormal"><span style="font-size:10.0pt;color:#003399">There are precedents for other drivers that depend on other software
(e.g. Oracle, I think), so I don't think anyone will strenuously object to your
driver being dependent on COM and ArcObjects. Obviously it would have to be an
optional driver. It would be good to write some tests for it; that may be a
requirement from the GDAL team, although I doubt they are in a position to run
them as part of a normal GDAL release, as I don't think any of them have
ArcGIS.</span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;color:#003399"> </span></p></div></div></blockquote><div>Yep, tests would be good. And also documentation :) </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div lang="EN-US" link="blue" vlink="purple"><div>
<p class="MsoNormal"><span style="font-size:10.0pt;color:#003399">A possible issue is that you'd probably need to recompile it
every time a different version of ArcGIS came out. Ideally, you would maintain
versions of it for each release of ArcGIS. The driver documentation should point
out to the user that they must obtain the right version. Alternatively you
might be able to craft one driver that dynamically determines the ArcGIS
version and does the right thing.</span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;color:#003399"> </span></p></div></div></blockquote><div>Are you volunteering for some help with the documentation? That would be nice ha ha.</div><div><br></div><div>Anyway, I don't suffer from being version dependent as long as ESRI does not change the CLSIDs of the COM objects I am using (they **rarely** do this and I cannot think of one example where they actually did change something *I* use). Being version dependent is true for .NET components, scripting and other languages that are tightly coupled with particular versions of dlls. This is not the case here.</div>
<div><br></div><div>Editing, Blob and timestamp fields are not implemented yet, but reading has been working fine for me.</div><div><br></div><div>I hope having FileGDB read support (even if it requires an ArcView or better license) will help some people.</div>
<div><br></div><div>Best Regards,</div><div><br></div><div>Ragi Yaser Burhum</div><div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div lang="EN-US" link="blue" vlink="purple">
<div><div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in"><p class="MsoNormal"><b><span style="font-size:10.0pt">From:</span></b><span style="font-size:10.0pt">
<a href="mailto:gdal-dev-bounces@lists.osgeo.org" target="_blank">gdal-dev-bounces@lists.osgeo.org</a> [mailto:<a href="mailto:gdal-dev-bounces@lists.osgeo.org" target="_blank">gdal-dev-bounces@lists.osgeo.org</a>] <b>On
Behalf Of </b>Ragi Y. Burhum<br>
<b>Sent:</b> Wednesday, January 13, 2010 12:53 PM<br>
<b>To:</b> <a href="mailto:gdal-dev@lists.osgeo.org" target="_blank">gdal-dev@lists.osgeo.org</a></span></p><div><div></div><div><br>
<b>Subject:</b> RE: [gdal-dev] Open source vector geoprocessing libraries?</div></div><p></p>
</div><div><div></div><div>
<p class="MsoNormal"> </p>
<div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<p class="MsoNormal" style="margin-bottom:12.0pt">Date: Wed, 13 Jan 2010 10:27:43
-0500<br>
From: "Jason Roberts" <<a href="mailto:jason.roberts@duke.edu" target="_blank">jason.roberts@duke.edu</a>><br>
Subject: RE: [gdal-dev] Open source vector geoprocessing libraries?<br>
To: "'Peter J Halls'" <<a href="mailto:P.Halls@york.ac.uk" target="_blank">P.Halls@york.ac.uk</a>><br>
Cc: 'gdal-dev' <<a href="mailto:gdal-dev@lists.osgeo.org" target="_blank">gdal-dev@lists.osgeo.org</a>><br>
Message-ID: <008001ca9464$f4059f10$dc10dd30$@<a href="mailto:roberts@duke.edu" target="_blank">roberts@duke.edu</a>><br>
Content-Type: text/plain; charset="US-ASCII"<br>
<br>
Peter,<br>
<br>
> are you constrained to retaining your data in an ArcGIS compatible format?<br>
<br>
<br>
We are attempting to build tools that can work with data stored in a variety<br>
of formats. Our current user community uses mostly shapefiles, ArcGIS<br>
personal geodatabases, and ArcGIS file geodatabases. Many of them are<br>
ecologists who do not have the interest or skills to deploy a real DBMS<br>
system. Thus we are hoping to provide tools that can work without one. This<br>
is one reason I was exploring how embeddable PostGIS and SpatiaLite might be<br>
in the other fork of this thread.<br>
<br>
> Until the File<br>
> Geodatabase format is published (later this year?) and someone has the<br>
effort to<br>
> build an OGR interface, the DBMS route is probably the best route to<br>
> compatibility.<br>
<br>
It would be really great for that to happen, but I'm not holding my breath.<br>
If it does get published, I would seriously contemplate building an OGR<br>
driver.<br>
<br>
I have contemplated building an ArcObjects- or arcgisscripting-based driver.<br>
This would at least allow people who have ArcGIS to use OGR to access any<br>
ArcGIS layer, including those created by ArcGIS's tools for joining<br>
arbitrary layers, etc. That would handle file geodatabases, as well as ALL<br>
formats accessible from ArcGIS. If such a driver existed, then we could use<br>
OGR as the base interface inside our application. But creating such a driver<br>
would be a lot of work and have funky dependencies because it either needs<br>
to use Windows COM (for ArcObjects) or Python (for arcgisscripting) to call<br>
the ArcGIS APIs. I am certainly capable of implementing it but because most<br>
of our code is in Python, it is probably easier for me to wrap OGR and<br>
arcgisscripting behind a common abstraction, and then have our tools work<br>
against that abstraction rather than OGR directly.</p>
</blockquote>
<div>
<p class="MsoNormal"> </p>
</div>
<div>
<p class="MsoNormal">I find it very amusing you mention this right now. </p>
</div>
<div>
<p class="MsoNormal"> </p>
</div>
<div>
<p class="MsoNormal">Why?</p>
</div>
<div>
<p class="MsoNormal"> </p>
</div>
<div>
<p class="MsoNormal">I asked Frank if there was an ArcObjects based OGR driver
this very past Thursday and he said "not that I know of". What I
wanted was, among other things, to get data out of FileGDB to PostGIS with one
shot and add some custom behavior for a client of mine. So I spent the
past three days looking at OGR drivers and wrote an ArcObjects based one. I got
it working yesterday.</p>
</div>
<div>
<p class="MsoNormal"> </p>
</div>
<div>
<p class="MsoNormal">- Right now I only instantiate 3 factories (Enterprise GDB
aka ArcSDE, AccessDB and FileGDB). This means it reads FileGDB just fine. If
you want more factories, the driver only has to be modified with one line to
add any other factories and everything else would just work.</p>
</div>
<div>
<p class="MsoNormal"> </p>
</div>
<div>
<p class="MsoNormal">- I only implemented the parts that I needed, so it is
readonly (should be straight forward to expand if need be). </p>
</div>
<div>
<p class="MsoNormal"> </p>
</div>
<div>
<p class="MsoNormal">- Although, it can read other GeoDatabase abstractions
(Topology, Geometric Networks, Annotations, Cadastral Fabrics, etc), currently
I am explicitly filtering for FeatureClasses and FeatureDatasets.</p>
</div>
<div>
<p class="MsoNormal"> </p>
</div>
<div>
<p class="MsoNormal">- It is a ATL / COM / C++ based one, so it will only compile
on Windows. It can be modified to use the cross platform ArcEngine SDK since
all the COM Objects that I use are called the same and behave the same way... I
just did not have an ArcEngine SDK installer, so I could not test this.</p>
</div>
<div>
<p class="MsoNormal"> </p>
</div>
<div>
<p class="MsoNormal">Anyway, if you are interested in the source code, let me
know. Perhaps we can add it as an ogr driver contribution (what is the process
for that anyway?). I may not respond fast enough to e-mail, since the next 4
weeks are pretty crazy for me.</p>
</div>
<div>
<p class="MsoNormal"> </p>
</div>
<div>
<p class="MsoNormal">- Ragi Burhum</p>
</div>
</div>
</div></div></div>
</div>
</blockquote></div><br>