[mapserver-users] Split large tiffs (topos) like TopoZone

Fri Oct 19 01:03:25 EDT 2001

Ed is quite correct (he should be, it's his site). I did the same thing for Minnesota (maps.dnr.state.mn.us/tomo, ~300,000 images) and it works very well. That script is simple perl script that doesn't use MapServer. However, I did use perl and MapScript to compute the 300k images (I think I still have the script if anyone want to take a look). Took a day. Can MapServer and the TOMO server work together? Possibly but you'd be bound by the non-MapServer application because of the nature of the underlying data.

Again, TOMO and TopoZone are built to serve one layer very fast, but I've been using the exact same datasets (quad-based DRGs) that the little images were computed from in a tiled fashion to support a much broader range of scales and applications with great success and by managing a few thousand images instead of a few hundred thousand. I can only imagine what the TopoZone folks go through with 17 million images and all the projection headaches.

Steve

Stephen Lime
Data & Applications Manager

Minnesota DNR
500 Lafayette Road
St. Paul, MN 55155
651-297-2937
>>> "Ed McNierney" <ed at topozone.com> 10/18/01 19:42 PM >>>
Folks -

I've already mentioned to John privately that I have set up a WMS server
for topographic maps and would be interested in working with folks who
would either like to use WMS "end-point" clients or use MapServer as a
WMS client for these data layers.  These will shortly be available as a
subscription service from TopoZone, and I could use some guinea pigs.
If you're very cooperative, I've got some nice 1-meter DOQ data you can
use, too <g>.  Send email if you're interested; you'll be able to get
complete DRG or DOQ base maps for your MapServer applications without
buying data or disk to put it on!

On the original point of chopping up rasters into little tiles, don't
get carried away.  TopoZone's current image architecture is designed to
do one very, very specific task very quickly.  The entire "map server"
for topozone.com (just the map-serving system, not the Web site) is six
lines of ASP VBScript code; everything is precomputed and prebuilt, and
all we serve are small image tiles at one of four fixed zoom levels.
The tiles are small so you can scroll the map in whole-tile increments
while still providing the user a reasonable degree of centering control.
The HARD part was generating all those tiles from the hodgepodge of
source DRG data - serving them is trivial.

I would NOT recommend a similar approach for MapServer applications -
the tiles are WAY too small.  Besides the administrative hassles (I've
got over 17 million of the little buggers to keep track of) the
disk-seeking overhead is not worth it.  You can construct a very nice
MapServer system using much larger files.  We're currently managing a
few applications with topographic base maps, are we're storing one quad
(one DRG) per file; that means each TIFF file is several megabytes.
There's NO problem with performance IF you structure the data carefully.
I could very easily use larger TIFF images, but there's no benefit for
me to do that - having one output TIFF image for every input DRG makes
updates, bug fixes, and administration much easier.

	- Ed

Ed McNierney
Chief Mapmaker
TopoZone.com

-----Original Message-----
From: Arthur, John [mailto:John.Arthur at voicestream.com]
Sent: Thursday, October 18, 2001 6:15 PM
To: 'kenboss'; Arthur, John
Cc: mapserver-users at lists.gis.umn.edu
Subject: RE: [mapserver-users] Split large tiffs (topos) like TopoZone

Hmm,.. I seams my mapserver won't display rasters,...
I'm using 3.5 w/php_mapscript on windows2000. It(phpinfo) says tiff was
compiled in, and I get no errors.
I'm using the ".wld" extention and also tried TileIndex through
shapefiles,.. Any ideas?

-John

-----Original Message-----
From: kenboss [mailto:kenboss at dilbert.dnr.state.mn.us]
Sent: Thursday, October 18, 2001 4:49 PM
To: John.Arthur at voicestream.com
Cc: mapserver-users at lists.gis.umn.edu
Subject: RE: [mapserver-users] Split large tiffs (topos) like TopoZone

Right, you'd lose the georeferencing with the ImageMagick approach.  If
you're a 
programmer, it shouldn't be too hard to do using one of the GDAL APIs
(C,
C++, 
or Python).  If you're only a wannabe programmer like myself, you might
still be 
able to hack one of the apps distributed with GDAL (such as
gdal_translate)
to 
get what you need.  See http://www.remotesensing.org/gdal/

--Ken
========================================================================
====
=
Ken Boss                                
Digital Image Analysis / Web Stuff       Forestry Resource Assessment
kenboss at dilbert.dnr.state.mn.us          Minnesota Dept. of Natural
Resources
Voice: 218 327 4449 ext. 237             413 SE 13th Street

Fax:   218 327 4517                      Grand Rapids, MN  55744
USA
                   www.ra.dnr.state.mn.us
========================================================================
====
=

> 
> Yes, but what about the world file(.twf)?
> 
> - John
> 
> -----Original Message-----
> From: Stephen Woodbridge [mailto:woodbri at mediaone.net]
> Sent: Thursday, October 18, 2001 4:21 PM
> To: Arthur, John
> Cc: Mapserver-Users at Lists. Gis. Umn. Edu (E-mail)
> Subject: Re: [mapserver-users] Split large tiffs (topos) like TopoZone
> 
> 
> I believe you could do that with ImageMagick and Perl to automate it.
> You should be able to chop up each of your large maps into n x m
tiles.
> 
> -Stephen Woodbridge
>  http://web-maps.org
> 
> "Arthur, John" wrote:
> > 
> > I would like to use topo quads in my MapServer, but it would take
forever
> to
> > export each quad from Delorme's 3D topoquads. So I exported large
150 mb
> > files instead.  Is there a way to split these up into smaller files
or
> just
> > to display the large tiff in MapServer faster.  We would like
something
> like
> > what TopoZone.com has (Mad props to you guys @ TopoZone.com), but
layer
> our
> > own data on top.
> > 
> > John Arthur
> > RF Technician
> > Office:   (757) 490-7221
> > Cellular: (757) 692-1515
> > Email: john.arthur at voicestream.com
> > _~-^-~_
> > VoiceStream
> > W I R E L E S S