[gdal-dev] Switching from ArcGIS to GDAL

[Smith, Michael]

Hi Jon - alas for we ArcGIS users, which even here in 2012 is still a
single-threaded application.  Who needs more than one CPU core anyway?
So your ArcGIS choice is to split your batch up into 12 groups and run
the processes simultaneously (which will then take all your CPUs), or
use something else.

Might I suggest a pretty simple solution which uses GDAL and MapServer
instead?  In the state of Maine we now serve all of our enterprise web
mapping services (WMS) for imagery using GDAL and MapServer.  Using
gdalwarp and gdal_translate we convert whatever the source imagery is
into JPEG2000 compressed tiles, and generate overview images using
gdalwarp.  This system takes our 3.5TB of source imagery down to about
700GB of JP2 files and is pretty darn fast.

You could set something like this up on a simple PC (but don't use a USB
drive for the data - put it on a fast internal drive) for a small group
of users inside your organization, or on servers for a wide distribution
to a large # of folks.

Our system is designed to support about 2000 end-users and about 10,000
requests per hour, using a dual-CPU (quad-core) HP server.  We use MS4W
which is super-easy to install, and then scripts to generate the tiles.
I'd be happy to share some of these with you as examples.


======================================
Michael Smith
State GIS Manager, Maine Office of GIS
State of Maine, Office of Information Technology
michael.smith _at_ maine.gov
207-215-5530

Board Member, Maine GeoLibrary
Board Member, Maine GIS Users Group
State Rep, National States Geographic Information Council

| MEGIS | GeoLibrary | MEGUG | NSGIC |

State House Station 174
264 Civic Center Drive
Augusta, ME 04333-0174

69 o 47' 49.5"W   44 o 20' 54.5"N 

Confidentiality Notice: This email message, including any attachments,
is for the sole use of the intended recipients and may contain
confidential and privileged information. If you are not the intended
recipient, or an authorized agent of the intended recipient, please
immediately contact the sender by reply email and destroy/delete all
copies of the original message. Any unauthorized review, use, copying,
forwarding, disclosure, or distribution by other than the intended
recipient or authorized agent is prohibited.

-----Original Message-----
From: gdal-dev-bounces at lists.osgeo.org
[mailto:gdal-dev-bounces at lists.osgeo.org] On Behalf Of jgamble
Sent: Saturday, February 25, 2012 1:21 PM
To: gdal-dev at lists.osgeo.org
Subject: [gdal-dev] Switching from ArcGIS to GDAL

Hi All,

I've recently been tasked to cache several large imagery datasets using
ArcGIS and have found that my bottleneck has been generating overviews
for
the datasets prior to caching.  ArcGIS appears to be limiting itself to
a
single core while building overviews, which seems ridiculous on a 12
core
machine ... which is why I am here for some help.

I am using mosaic datasets in ArcGIS to manage my images.  Footprints
for
the images have been created to clip out any bounding black pixels.  Am
I
able to define overviews in ArcGIS and then use GDAL to build them with
multiple cores ... Or at least utilize the footprints I've already
defined
in my mosaic dataset?  Does GDAL have something similar to a mosaic
dataset
to manage images that I should be using?

I'm extremely new to the thought of using GDAL, so any advice on how to
go
about utilizing it or at least pointing me in the right direction to
some
tutorials is much appreciated.

Thanks
Jon.      

--
View this message in context:
http://osgeo-org.1560.n6.nabble.com/Switching-from-ArcGIS-to-GDAL-tp4507
244p4507244.html
Sent from the GDAL - Dev mailing list archive at Nabble.com.
_______________________________________________
gdal-dev mailing list
gdal-dev at lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev