[mapserver-users] Dual role of LAYER is confusing
Rahkonen Jukka (Tike)
jukka.rahkonen at mmmtike.fi
Tue Feb 11 06:18:51 PST 2014
Hi,
Yes, I do use use INCLUDEs and I can find easily fine, awkward stuff like this from my mapfiles.
INCLUDE "/data/maps/sp_tileindex/spatialite_tileindex.map"
INCLUDE "/data/maps/sp_tileindex/spatialite_tileindex_2013.map"
INCLUDE "/data/maps/sp_tileindex/spatialite_tileindex_2012.map"
INCLUDE "/data/maps/sp_tileindex/spatialite_tileindex_2011.map"
INCLUDE "/data/maps/sp_tileindex/spatialite_tileindex_2010.map"
INCLUDE "/data/maps/sp_tileindex/spatialite_tileindex_2009.map"
INCLUDE "/data/maps/sp_tileindex/spatialite_tileindex_2008.map"
INCLUDE "/data/maps/sp_tileindex/spatialite_tileindex_2007.map"
INCLUDE "/data/maps/sp_tileindex/spatialite_tileindex_2006.map"
INCLUDE "/data/maps/sp_tileindex/spatialite_tileindex_2005.map"
INCLUDE "/data/maps/sp_tileindex/spatialite_tileindex_2004.map"
INCLUDE "/data/maps/sp_tileindex/spatialite_tileindex_2003.map"
INCLUDE "/data/maps/sp_tileindex/spatialite_tileindex_2002.map"
INCLUDE "/data/maps/sp_tileindex/spatialite_tileindex_2001.map"
INCLUDE "/data/maps/sp_tileindex/spatialite_tileindex_2000.map"
The question is "Is this the way you would like to learn your mother to configure you a few new layers?". But I don't know her, she can be a nerd. How about your brother?
-Jukka-
Stephen Woodbridge wrote:
> Hi Jukka,
>
> You already have this in a fashion.
>
> Your DATADEFSET is a folder.
> And you DATADEF is called INCLUDE
>
> So create a layer_name.def files for each of your layer data sources INCLUDE
> "datadefset/layer_name.def"
> in your layer definitions.
>
> -Steve W
>
> On 2/11/2014 7:30 AM, Rahkonen Jukka (Tike) wrote:
> > Hi,
> >
> > I will change my course to direction "Defining input data for Mapserver is
> confusing". Dual role of LAYER is just a detail and using a layer as input for
> heatmap generator feels actually rather good. But there is something more
> general that smells rotten to me and I cannot be the most negative people to
> say so. After all, I do know something about mapfiles, I maintain a few hundred
> WMS layers and I can myself do many tricky things very effectively with "copy-
> paste-edit" and "find-replace" and with heavy use of INCLUDEs. I can say that
> the system is powerful, flexible and maintainable - for me. And each summer I
> have experienced that it is impossible to make the one who should substitute me
> during my holidays to believe me.
> >
> > I try to explain why defining the input data for Mapserver feels sometimes
> awkward. As a user, when I create a new vector layer all the input data feel like
> the same. After all, they are just vectors and I do not care if they come from a
> shapefile, database, or tileindex. When there are hundreds of layers to
> maintain, it would be very nice if:
> >
> > - Defining the input data would follow always the same logic, even
> > better if also the same syntax
> > - Input data definition could be put on one line so it would be simple
> > to edit the whole mapfile with a text editor and search-replace
> > - Changing from shapefile to using a bunch of files through
> > ogrtileindex or to a database could be done with one simple edit
> > - Input data definition would be reusable over many layers and mapfiles.
> >
> > Let's have a look and see how we define the input vector data now. Myself I
> need to use only 6 of these methods, the others I collected from Mapserver
> documentation. Naturally I use raster data too and that makes 4 input types
> more (file, shapefile tileindex, spatialite tileindex, WMS) but those feel rather
> convenient for me. So these are the vector data input methods with usage
> examples.
> >
> > 1) Shapefile:
> > DATA "shapefile"
> >
> > 2) Native shapefile tileindex:
> > TILEINDEX "shapefile"
> > TILEITEM "LOCATION"
> >
> > 3) Tileindex through OGR
> > CONNECTIONTYPE OGR
> > TILEINDEX "/shapefile.shp,0"
> > TILEITEM "LOCATION"
> >
> > 4) Tileindex from a layer, which must be defined separately TILEINDEX
> > "layer"
> >
> > 5) Connection through OGR, in this case to Spatialite database
> > CONNECTIONTYPE OGR CONNECTION "/spatialite.sqlite"
> > DATA "select * from layer"
> >
> > 6) Native SDE-connection
> > CONNECTION "sdemachine.iastate.edu,port:5151,sde,username,password"
> > CONNECTIONTYPE SDE
> > DATA "HOBU.STATES_LAYER,SHAPE,SDE.DEFAULT"
> > FILTER "where MYCOLUMN is not NULL"
> >
> > 7) Connection through a plugin
> > CONNECTIONTYPE PLUGIN
> > PLUGIN "msplugin_mssql2008.dll"
> > CONNECTION "Server=.\MSSQLSERVER2008;Database=Maps;Integrated
> Security=true"
> > DATA "ogr_geometry from rivers USING UNIQUE ogr_fid USING SRID=4326"
> >
> > 8) Native Oracle connection
> > CONNECTIONTYPE oraclespatial
> > DATA "MYGEOMETRY FROM MYTABLE USING UNIQUE MYTABLE_ID"
> >
> > 9) Native PostGIS connection
> > CONNECTIONTYPE POSTGIS
> > CONNECTION "host=yourhostname dbname=yourdatabasename
> user=yourdbusername
> > password=yourdbpassword port=yourpgport"
> > DATA "geometrycolumn from yourtablename"
> >
> > 10) Connection to remote WFS service
> > CONNECTION "http://demo.mapserver.org/cgi-bin/wfs?"
> > CONNECTIONTYPE WFS
> > METADATA
> > "wfs_typename" "continents"
> > "wfs_version" "1.0.0"
> > "wfs_connectiontimeout" "60"
> > "wfs_maxfeatures" "10"
> > END
> >
> >
> > I do not believe that there was a bizarre developer who made a plan about
> implementing all these data input systems for Mapserver v. 0.1. Rather they
> have just appeared one after another during the years.
> >
> > As a user I can't say what might work and what wouldn’t so I can only
> challenge the developers to think how to make data input more user friendly.
> However, I do have some thoughts.
> > - Individual layers are not so hard to configure but it is irritating that there are
> so many different configurations.
> > - Fortunately so many formats are supported only through OGR which gives
> the same syntax for all: CONNECTIONTYPE + CONNECTION + DATA.
> > - I wonder why DATA was not originally defined as block like many analogous
> things like PROJECTION, SYMBOL, CLASS, STYLE were. Perhaps nobody thought
> about anything else than shapefiles.
> > - We may introduce a new object "DATADEFINITION" that holds everything
> that is needed to read the input vectors: path to shapefile or
> CONNECTIONTYPE+CONNECTION+DATA or ogrtileindex path.
> > - Like SYMBOL, DATADEFINITION (or shorter, DATADEF) could have a name.
> > - DATADEF would be defined like a symbol
> > DATADEF
> > NAME "streetdata"
> > CONNECTIONTYPE OGR
> > CONNECTION "/data/osm_sqlite"
> > DATA "select * from lines where highway is not null"
> > END
> > - DATADEF could be defined it the mapfile and reused by its name.
> > - In LAYER, one line would by enough for defining the input data source but
> support for old syntax remains.
> > LAYER
> > DATADEF "streetdata"
> > ......
> > - We have SYMBOLSET and FONTSET and for the same reasons we could have
> DATADEFSET file. DETADEFs stored into the DATADEFSET file could be reused
> across multiple mapfiles. Hey, only one or handful of places to update after
> password change! In practice that would make it possible to change the
> passwords.
> > Hmm, it should be possible also to encrypt the DATADEFSET file but I leave that
> to the developers.
> > - I am not sure if it would work, but perhaps it could be possible to separate, if
> needed, DATADEF and DATADEFFILTER. There is an example in the native SDE
> connector which seems to have a FILTER. For example for OGR layers it would
> mean storing only CONNECTIONTYPE and CONNECTION into DATADEF. SQL
> select could come from DATADEFFILTER. It would give a possibility to re-use a
> common database connection in a mapfile but create layers by adding one line
> for unique selects. Unfortunately the DATADEFFILTER would probably not work
> if DATADEF is changed to read data from a shapefile instead of PostGIS db.
> Another problem for the developers.
> >
> > -Jukka Rahkonen-
> >
> > thomas bonfort wrote:
> >> On 10 February 2014 15:13, Rahkonen Jukka (Tike)
> >> <jukka.rahkonen at mmmtike.fi> wrote:
> >>> Hi,
> >>>
> >>> In most cases in Mapfile LAYER is defining the output of the
> >>> service. However,
> >> it looks like the original way to use DATA has not been flexible
> >> enough to suit new sources of data and as a workaround one layer is
> >> used as an input for another layer. For example if a shapefile is
> >> used as tileindex it is enough to write TILEINDEX "tiger/index.shp"
> >>> However, if one wants to take tileindex from Spatialite it must be
> >>> done by defining a layer first
> >>>
> >>> LAYER # The tileindex layer
> >>> NAME "spatialite_tileindex"
> >>> STATUS OFF
> >>> TYPE POLYGON
> >>> CONNECTIONTYPE OGR
> >>> CONNECTION "/orthophotoindex.sqlite"
> >>> DATA "select * from orthophotos where year=2013"
> >>> PROJECTION
> >>> "init=epsg:3067"
> >>> END
> >>> END # End of tileindex layer
> >>> LAYER # The orthophoto layer
> >>> NAME "orthophotos_2013 "
> >>> STATUS ON
> >>> TILEINDEX "spatialite_tileindex" # Name of the tileindex
> >>> layer ....
> >>>
> >>> I guarantee that many if not all new Mapserver users consider this as tricky.
> >> However, situation can be even more tricky. The new RFC about
> >> heatmaps/density maps gives an example
> >> http://mapserver.org/development/rfc/ms-rfc-108.html.
> >>> The source data for the heatmap layer is configured as
> >>>
> >>> LAYER
> >>> NAME "heatmap"
> >>> TYPE raster
> >>> CONNECTIONTYPE kerneldensity
> >>> CONNECTION "points"
> >>>
> >>> and here "points" is referring to a layer. It the "points" layer is
> >>> using
> >> ogrtileindex where tileindex is not a shapefile but comes from a
> >> database we will need three layers for publishing the heatmap:
> >>> - ogrtileindex layer
> >>> - "points" layer
> >>> - heatmap layer
> >>>
> >>> This is doable but confusing. What is also confusing is that I have
> >>> not yet found
> >> a way to hide those technical layers from WMS.
> >>
> >> Supposing you had that use-case given your data files, how would you
> >> specify it in a more readable way if you didn't have this syntax?
> >
> > > Until now after making a web search during a coffee break, it is
> >>> naturally done with a layer level metadata item "ows_enable_request"
> >>> "!*" as described in
> >>> http://mapserver.org/de/development/rfc/ms-rfc-67.html. Great. But
> >>> still I feel the dual role of "layer" a bit confusing. At least in
> >>> the tileindex case I wish I could simply write just one line even if
> >>> with somehow complicated syntax. Perhaps it could be something like
> >>>
> >>> TILEINDEX "SELECT * from orthophotos where year=2013 USING OGR
> >> CONNECTION /orthophotoindex.sqlite"
> >>
> >> I'd argue that referencing an existing layer is less confusing. In
> >> any case it is more powerfull, more flexible and more maintainable in our
> codebase.
> >
> >> Interesting points nevertheless....
> >
> > -Jukka-
> >
> >> best regards,
> >> thomas
> >>
> >>>
> >>> -Jukka Rahkonen-
> >>>
> >>>
> >>>
> >>>
> >>> _______________________________________________
> >>> mapserver-users mailing list
> >>> mapserver-users at lists.osgeo.org
> >>> http://lists.osgeo.org/mailman/listinfo/mapserver-users
> > _______________________________________________
> > mapserver-users mailing list
> > mapserver-users at lists.osgeo.org
> > http://lists.osgeo.org/mailman/listinfo/mapserver-users
> >
>
> _______________________________________________
> mapserver-users mailing list
> mapserver-users at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/mapserver-users
More information about the mapserver-users
mailing list