[mapserver-users] Split large tiffs (topos) like TopoZone
Allan Doyle
adoyle at intl-interfaces.com
Fri Oct 19 08:22:05 EDT 2001
I think the original question/need was to be able to have DRGs "like
TopoZone's" that John Arthur could layer his own data on top of.
If, indeed, he meant that he wants to layer his data on top of
TopoZone's data, then since we learned (also in this thread) that
TopoZone has a WMS, then he's home free. The idea behind WMS was to
allow precisely this: "Map A" from source "A" can be layered onto "Map
B" from source "B" without having to copy all the data from "A" to "B"
or vice versa. This can be done at the client end using CSS tricks with
layers - i.e. the client sends out two requests, one to WMS A and one to
WMS B and can layer the result. Or it can be done using "Cascading WMS"
techniques - WMS A behaves as a client to WMS B, pulls WMS B's layers in
on demand, and produces the layered map for WMS A.
http://viewer.digitalearth.gov/ is a great example of the CSS approach.
http://www.cubewerx.com/demo/cubeview/cubeview.cgi is the original and
probably the best current example of the cascading approach.
Also, not to be heretical in the mapserver-users list, but there's an
open source implementation of a WMS geared to displaying from large
image bases called the MIT OrthoServer at
http://tull.mit.edu/orthoserver/ You can use it for the image layers
and the mapserver for the other layers and cascade up through the
mapserver.
Allan
Stephen Lime wrote:
>
> Ed is quite correct (he should be, it's his site). I did the same thing for Minnesota (maps.dnr.state.mn.us/tomo, ~300,000 images) and it works very well. That script is simple perl script that doesn't use MapServer. However, I did use perl and MapScript to compute the 300k images (I think I still have the script if anyone want to take a look). Took a day. Can MapServer and the TOMO server work together? Possibly but you'd be bound by the non-MapServer application because of the nature of the underlying data.
>
> Again, TOMO and TopoZone are built to serve one layer very fast, but I've been using the exact same datasets (quad-based DRGs) that the little images were computed from in a tiled fashion to support a much broader range of scales and applications with great success and by managing a few thousand images instead of a few hundred thousand. I can only imagine what the TopoZone folks go through with 17 million images and all the projection headaches.
>
> Steve
>
> Stephen Lime
> Data & Applications Manager
>
> Minnesota DNR
> 500 Lafayette Road
> St. Paul, MN 55155
> 651-297-2937
> >>> "Ed McNierney" <ed at topozone.com> 10/18/01 19:42 PM >>>
> Folks -
>
> I've already mentioned to John privately that I have set up a WMS server
> for topographic maps and would be interested in working with folks who
> would either like to use WMS "end-point" clients or use MapServer as a
> WMS client for these data layers. These will shortly be available as a
> subscription service from TopoZone, and I could use some guinea pigs.
> If you're very cooperative, I've got some nice 1-meter DOQ data you can
> use, too <g>. Send email if you're interested; you'll be able to get
> complete DRG or DOQ base maps for your MapServer applications without
> buying data or disk to put it on!
>
> On the original point of chopping up rasters into little tiles, don't
> get carried away. TopoZone's current image architecture is designed to
> do one very, very specific task very quickly. The entire "map server"
> for topozone.com (just the map-serving system, not the Web site) is six
> lines of ASP VBScript code; everything is precomputed and prebuilt, and
> all we serve are small image tiles at one of four fixed zoom levels.
> The tiles are small so you can scroll the map in whole-tile increments
> while still providing the user a reasonable degree of centering control.
> The HARD part was generating all those tiles from the hodgepodge of
> source DRG data - serving them is trivial.
>
> I would NOT recommend a similar approach for MapServer applications -
> the tiles are WAY too small. Besides the administrative hassles (I've
> got over 17 million of the little buggers to keep track of) the
> disk-seeking overhead is not worth it. You can construct a very nice
> MapServer system using much larger files. We're currently managing a
> few applications with topographic base maps, are we're storing one quad
> (one DRG) per file; that means each TIFF file is several megabytes.
> There's NO problem with performance IF you structure the data carefully.
> I could very easily use larger TIFF images, but there's no benefit for
> me to do that - having one output TIFF image for every input DRG makes
> updates, bug fixes, and administration much easier.
>
> - Ed
>
> Ed McNierney
> Chief Mapmaker
> TopoZone.com
>
> -----Original Message-----
> From: Arthur, John [mailto:John.Arthur at voicestream.com]
> Sent: Thursday, October 18, 2001 6:15 PM
> To: 'kenboss'; Arthur, John
> Cc: mapserver-users at lists.gis.umn.edu
> Subject: RE: [mapserver-users] Split large tiffs (topos) like TopoZone
>
> Hmm,.. I seams my mapserver won't display rasters,...
> I'm using 3.5 w/php_mapscript on windows2000. It(phpinfo) says tiff was
> compiled in, and I get no errors.
> I'm using the ".wld" extention and also tried TileIndex through
> shapefiles,.. Any ideas?
>
> -John
>
> -----Original Message-----
> From: kenboss [mailto:kenboss at dilbert.dnr.state.mn.us]
> Sent: Thursday, October 18, 2001 4:49 PM
> To: John.Arthur at voicestream.com
> Cc: mapserver-users at lists.gis.umn.edu
> Subject: RE: [mapserver-users] Split large tiffs (topos) like TopoZone
>
> Right, you'd lose the georeferencing with the ImageMagick approach. If
> you're a
> programmer, it shouldn't be too hard to do using one of the GDAL APIs
> (C,
> C++,
> or Python). If you're only a wannabe programmer like myself, you might
> still be
> able to hack one of the apps distributed with GDAL (such as
> gdal_translate)
> to
> get what you need. See http://www.remotesensing.org/gdal/
>
> --Ken
> ========================================================================
> ====
> =
> Ken Boss
> Digital Image Analysis / Web Stuff Forestry Resource Assessment
> kenboss at dilbert.dnr.state.mn.us Minnesota Dept. of Natural
> Resources
> Voice: 218 327 4449 ext. 237 413 SE 13th Street
>
> Fax: 218 327 4517 Grand Rapids, MN 55744
> USA
> www.ra.dnr.state.mn.us
> ========================================================================
> ====
> =
>
> >
> > Yes, but what about the world file(.twf)?
> >
> > - John
> >
> > -----Original Message-----
> > From: Stephen Woodbridge [mailto:woodbri at mediaone.net]
> > Sent: Thursday, October 18, 2001 4:21 PM
> > To: Arthur, John
> > Cc: Mapserver-Users at Lists. Gis. Umn. Edu (E-mail)
> > Subject: Re: [mapserver-users] Split large tiffs (topos) like TopoZone
> >
> >
> > I believe you could do that with ImageMagick and Perl to automate it.
> > You should be able to chop up each of your large maps into n x m
> tiles.
> >
> > -Stephen Woodbridge
> > http://web-maps.org
> >
> > "Arthur, John" wrote:
> > >
> > > I would like to use topo quads in my MapServer, but it would take
> forever
> > to
> > > export each quad from Delorme's 3D topoquads. So I exported large
> 150 mb
> > > files instead. Is there a way to split these up into smaller files
> or
> > just
> > > to display the large tiff in MapServer faster. We would like
> something
> > like
> > > what TopoZone.com has (Mad props to you guys @ TopoZone.com), but
> layer
> > our
> > > own data on top.
> > >
> > > John Arthur
> > > RF Technician
> > > Office: (757) 490-7221
> > > Cellular: (757) 692-1515
> > > Email: john.arthur at voicestream.com
> > > _~-^-~_
> > > VoiceStream
> > > W I R E L E S S
--
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Allan Doyle http://www.intl-interfaces.com
adoyle at intl-interfaces.com
More information about the mapserver-users
mailing list