[Mapserver-dev] Generic tiling (for shapefiles and rasters)...

[Steve Lime]

> Hi Steve,

Hi Steve.

> We talked about the need to improve on the tile4ms to validate the 
> all the files being added to the tileindex have the same fields and 
> order of fields. What about creating a tileindex.mdt file that would 
> be a metadata file with the template info for the tileindex. This 
> file can be created by tile4ms and be a simple text file that anyone 
> anyone can create that has the dbf definition that was validated when 
> tile4 ms was run.

> This file could be opened if the tileindex is a shapfile instead of 
> scanning the files. This would imply that for non shapfile 
> tileindexes that you can request a similar metadata structure for the 
> layer.

> Another option would be to require the meta data to included in the 
> METADATA ...END object of the LAYER in the mapfile is the data layer 
> is shapfiles.

I think this is still a good idea, updating tile4ms that is. I really
can't 
see non-shapefile indexes for shapefiles, not in the near future.

> How does this play with the fact that I build tileindexes for Tiger 
> data based on the county directory and not on the actual files in the 
> layer. So the data is organized like this:

> SHAPEPATH "/data/mdata/"
> LAYER
>   TILEINDEX "tiger-tile"
>   DATA "roads"
> END
> LAYER
>   TILEINDEX "tiger-tile"
>   DATA "waterpoly"
> END

> the LOCATION in the tileindex look like
> /data/mdata/tiger2k/25/25015/
> /data/mdata/tiger2k/25/25017/
> etc

> and 

> files are like:

> /data/mdata/tiger2k/25/25015/roads.*
> /data/mdata/tiger2k/25/25015/waterpoly.*
> /data/mdata/tiger2k/25/25017roads.*
> /data/mdata/tiger2k/25/25017/waterpoly.*
> etc

> This cuts down on the number of records in the tileindex from about 
> 33,000 to about 3300 at the minor expense that some layers are 
> marginally smaller than the county bounds, but the biggest layers 
> "roads" always defines the boundary anyway. and drives performance.

The modifications I'm talking about should have no effect on file-based
versus directory-based access. The location (or TILEITEM) contents are
applied to the layer that is tiled regardless of the where the tileindex
is defined. Your old mapfile would still work. You could re-write as

LAYER
  NAME tiger-tile
  DATA tiger-tile
END
LAYER
   TILEINDEX "tiger-tile"
   DATA "roads"
END
LAYER
  TILEINDEX "tiger-tile"
  DATA "waterpoly"
END

A bit more verbose but now you can do filtering within your tiles. For 
example, you could take a single county out of production. Or you
could store multiple years worth of data within the same tile and could
specify a particular year at runtime.

Will you be willing to test?

> -Steve W.

Steve

On 8 Mar 2004 at 17:06, Steve Lime wrote:

> Hi Folks: As I've been putzing with enhanced tiling support I've been
> trying to generalize things
> so that one wouldn't be limited to simply shapefile tile indexes. For
> raster data that shouldn't be
> an issue using another LAYER as a tile layer. This would allow you,
for
> example, to store your
> tileindex in PostGIS and reference images on the file system. For
> shapefiles it's not that simple
> though because of this need to open one tile as a template (to get
item
> names and such). I can'd
> find an easy way to retrieve, for example, the 1st record in a layer.
> (For rasters this isn't an 
> issue since we don't need a template.)
> 
> So, there are a couple of options:
> 
>   - say the hell with non-shapefile tile indexes for shapefiles, but
> allow them for rasters
> 
>   - figure out a method to retrieve the nth element of a datasource
> (not by ID, but by position)
> 
>   - retool the msLayer... functions and move the msLayerWhichItems
> functionality someplace else
> 
> I'm partial to the first one. I can't imagine anyone using
> non-shapefile tile indexes with shapefiles 
> (I mean, most folks would put everything in PostGIS or SDE). Raster
> data are where that functionality
> makes sense.
> 
> I just want to run the idea by folks to make sure I'm not overlooking
> some obvious solution.
> 
> Steve
> _______________________________________________
> Mapserver-dev mailing list
> Mapserver-dev at lists.gis.umn.edu
> http://lists.gis.umn.edu/mailman/listinfo/mapserver-dev
> 




Stephen Lime
Data & Applications Manager

Minnesota DNR
500 Lafayette Road
St. Paul, MN 55155
651-297-2937