[webmap-discuss] OGC and google style tiling
Paul Spencer
pspencer at dmsolutions.ca
Thu Apr 27 16:08:56 EDT 2006
Steven,
(sorry folks, this is getting longer and longer)
On 27-Apr-06, at 10:00 AM, Steven M. Ottens wrote:
> Paul Spencer wrote:
>> Steven
>>
>> there are a couple of very good reasons for this (in my opinion).
>> The
>> approach you suggest is actually the first one I tried because I was
>> focussing on the client side code at the time. It became obvious
>> that
>> it would not actually work in a real world situation because:
>>
>> 1) every request to mapserv starts the CGI, loads the map, renders
>> the
>> map and returns it. This takes a substantial amount of time, even if
>> your map file and data are highly optimized. For instance, my test
>> cases would take about 1-2 seconds per tile, with a HUGE load on the
>> server.
> The idea is to get serverside cache, we have caching working on
> mapserver.cgi requests, which means it can can deliver lots and
> lots of
> tiles in a second. The reason I'm developing tiling is because our
> current setup can only handle 20 concurrent users and we need 300.
> If we
> can server from cache we can do that, and tiling is a way to make sure
> identical images are served to users and as such can be served from
> cache.
> Also in your situation you need to run a tile.php, a mapserver.cgi
> and a
> gd (why btw?) image process. If you multiply this 300 times it's also
> quite a load on the server. I prefer to keep as much load on the
> client
> side as possible.
how are you doing the serverside cache on mapserv requests?
In the ka-Map case, tile.php only loads mapscript and gd if the tile
has not been previously generated. Tiles are genereated the first
time they are requested and subsequently stored on disk and returned
directly by tile.php if they already exists
GD is required because we generate 'metatiles', typically 25-36 tiles
constituting a large map image (1000-2000 pixels square), in a single
map draw and slice it up into smaller tiles using GD after. The
large map draw minimizes the number of times we need to actually load
mapscript and render data, and also minimizes the effect of the label/
symbol/rendering issues.
>>
>> 2) every request to mapserv is treated as a single map draw. This
>> means that all label locations are calculated for that tile only. In
>> many cases, labels are too big to fit on a single tile. In others,
>> you have a label that fills the tile and because the feature extends
>> across multiple tiles, you may get the identical label repeated next
>> to each other many times.
> that is a problem indeed :( But if you keep a reasonable tile size
> it's
> not too disturbing.
ka-Map uses a tile size of 200x200 in its default, but I have started
switching to 300x300. In either case, the tile size minimizes the
number of actual images that are loaded (which impacts browser
performance when dragging the map) while keeping the download size
reasonable for each tile and providing a buffer of tiles 'offscreen'
so that users should rarely see empty tiles menu panning.
-- By comparison, google uses 256x256 (they used 128x128 for a while
then changed)
>>
>> 3) mapserv has rendering issues that are difficult to detect when
>> rendering a single image, but when you render tiles, they become very
>> obvious. The worst is thick lines rendered with a 'circle' symbol,
>> the ends of the line (at the map draw boundary) are tailed off ...
>> when two tiles come together, there is a very noticeable 'shearing'
>> effect on the lines which looks very bad.
> bad, bad, mapserver ;) I noticed the glitches after careful
> inspection.
> So either our data is not sensitive to this error or you have
> better eyes ;)
It is most noticeable when rendering thick lines 10+ pixels wide. I
had never noticed it before I started doing the tiling. When I first
saw it, I actually though the that tiles were mis-aligned by a few
pixels :) I spent quite a bit of time playing with my code before I
figured out I was right and mapserver was wrong ...
>>
>> 4) it is not possible to modify the headers returned from the cgi,
>> which means that you get the default browser caching behaviour. In
>> many cases, tiles can be cached for a long time, and in some specific
>> cases you want no caching at all.
> You can modify headers with apache, we set expiry dates for all of our
> mapimages. In our case the data is not very dynamic so a expiry
> date of
> a day or even a week is fine.
ok
>>
>> 5) I wanted to have some way of providing the capability to switch
>> between maps. This requires some knowledge of what maps are
>> available
>> on the server. There is no way to do this with the CGI
> That's not a real tiling issue, is it? In mapbuilder you can create a
> list of available layers based on the getCapabilitiesDoc. I use
> that to
> provide alternative layers.
I didn't want ka-Map to be tied to ogc since not that many folks
outside our community care about ogc.
>>
>> 6) in ka-Map (as with Google), you are restricted to a specific
>> set of
>> scales for viewing the map. This is not strictly necessary if you
>> are
>> not caching the tiles, but in the case of cached tiles it does become
>> very important. Cartographically, this means you can build your map
>> file for very specific scale breaks and ensure that the map looks
>> exactly right at each of the scales. Using just the cgi, it would be
>> impossible to determine the intended scale breaks for a given map.
> The getCapabilitiesDoc provides minimum and maximum scales so they can
> be used to determine the scale levels. Alternatively you will need to
> configure these scale levels in the configuration file. I do see an
> issue here, but I don't think it's a real showstopper.
agreed that its not a show stopper
>>
>> On a related note, however, if you wanted to try this anyway there is
>> a wmsLayer type in kaMap that can be used to create layers based on
>> remote WMS sources on the fly. It would be a useful exercise to
>> remove the server-side dependency in ka-Map's initialization step and
>> only trigger it on demand so that an 'empty' map could be created to
>> which you could add some number of wms layers. This would be a very
>> good fit as it would remove the server-side dependency and, as
>> MapBuilder is already very ogc-aware, it would be easy to integrate.
> The empty map is the one with the aPixel.src which I find every now
> and
> then in the ka-map code? I'm still trying to understand kamaps
> approach
> to building the tiles. Do you mean that you create a empty 'grid' of
> tiles, where you attach the layers on?
ka-Map has several objects that participate in this ...
kaMap - this is a mixture of several different concepts (and probably
needs to be refactored), but essentially it manages the user
interface by creating a DIV inside the user-specified element that
contains spatially positioned elements - in this case other DIVs.
The spatially positioned elements are absolutely positioned inside
their containing DIVs in a deterministic way (spatial location *
geographicUnitsPerPixel). The top/left style of the container is
changed to 'move' the contained elements. The logic for determining
if new tiles need to be loaded is also in here.
kaTool - base class for kaNavigator and other tools that provide user
controls such as dragging the map. These classes intercept events
from the user and programmatically adjust the top/left of the
container to move the tiles (for instance)
_map - encapsulates the concept of a map that contains layers. It
maintains information specific to a map such as the layers, the
current extents, default extents, maximum extents, background color etc.
_layer - encapsulates the concept of a layer, including name,
visibility, opacity etc. _layer is actually used to set the src of
each image that it contains, allowing for new _layer classes to
request tiles in different ways - I used this to implement a wms
layer that translates the pixel location of each tile into a wms
request rather than a call to tile.php.
By default, when you initialize ka-Map, it executes an ajax request
to init.php which reads config.php and returns javascript objects for
each _map and _layer and triggers the initial _map selection (either
the default from the config.php or the one requested by the user
calling the initialize function. This process could easily be
deferred, allowing the application to build its own _map instance in
the client from a context document. This is what I meant by 'empty
map' - poorly worded, sorry.
If we removed the requirement for calling init.php then you would
have no server-side dependency at all.
-- I just checked how it works and calling initialize is actually not
done in the constructor for kaMap so it should be possible to do this
already. I've attached an html file which uses kaMap to render a wms
layer with no server side interaction at all. You will need to
update kaMap to the latest cvs version because wmsLayer was slightly
out of date with a change I made to the _layer constructor.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.osgeo.org/pipermail/mail_webmap-discuss/attachments/20060427/474d20a9/mbtest.html
-------------- next part --------------
>>
>> I may be willing to invest some effort in getting this to work at
>> some
>> point, especially if there is some interest/cooperation from the
>> MapBuilder community (which has already been offered btw, thanks Mike
>> and Cameron)
> I am the one from the mapbuilder community who is working on tiling ;)
> We (the mb community) are interested in tiling, be it Ka-map style,
> openlayers-style or our own implementation. I prefer cooperation with
> other webmapping projects, instead of our own implementation.
>
> I was brainstorming with Cameron on this topic and I can up with an
> alternative approach to tiling. Based on your concerns on the edges
> and
> the labels (also the performance, but that can be solved by serverside
> caching): Why not extend the WMS standard with a &tile=true parameter.
> It will be one request, but mapserver will create 'absolute' tiles and
> set a set of tiles back.
> This requires modification on both the WMS server and the WMS
> client and
> as such not easily done, but I think it's in the long run a better
> solution. Meanwhile I'll continue on a pure clientside tiling
> mechanism.
> We will probably discuss it at the mapbuilder meeting of next week.
> Anyone interested in this topic is free to join.
I believe that a tiling extension to the spec is in the works? Until
WMS servers catch up, I'll stick with what we've got :)
Cheers
Paul
>
> Steven
>
>>
>> Cheers
>>
>> Paul
>>
>> On 27-Apr-06, at 4:32 AM, Steven M. Ottens wrote:
>>
>>> Hi Paul,
>>>
>>> Why did you choose to use a php script to handle tiling? It makes it
>>> much harder for ka-map to become a general webmapping client,
>>> currently
>>> it is strongly tied to mapserver. Also for me it's more difficult to
>>> integrate it with mapbuilder, if I want to keep mapbuilder general.
>>> Why not integrate the routine in the clientside and request tiles
>>> like this:
>>> http://www.edugis.nl/cgi-bin/edugis/mapserv.cgi?map=maps/edugis/
>>> base.map&VERSION=1.1.0&REQUEST=GetMap&LAYERS=topografie&STYLES=defau
>>> lt&SRS=EPSG:
>>> 4326&BBOX=4.833984375,52.26815737376817,4.921875,52.32191088594773&W
>>> IDTH=256&HEIGHT=256&FORMAT=image/
>>> png&TRANSPARENT=TRUE&EXCEPTIONS=INIMAGE
>>>
>>> (this is a request from a gmap-based application)
>>> The important part, obviously, to make sure that the tiles have
>>> the same
>>> size (easy) and the same coordinates regardless of the client.
>>>
>>>
>>> grt,
>>> Steven
>>>
>>>
>>>
>>> Paul Spencer wrote:
>>>> Ok. I'm not sure if the URLs to the tiles are standard. The
>>>> URL goes
>>>> to a tile.php script which converts special pixel coordinates into
>>>> geographic coordinates. The math for the conversion is very simple
>>>> though, all you need to know is the size of a pixel in geographic
>>>> coordinates.
>>>>
>>>> I'll need to look into mapbuilders controls to understand how to do
>>>> the integration. Perhaps this could be the start of a common
>>>> API? ;)
>>>>
>>>> Cheers
>>>>
>>>> Paul
>>>>
>>>> On 13-Apr-06, at 11:33 PM, Cameron Shorter wrote:
>>>>
>>>>> The "standard URL" I was refering to is the URL of a tile.
>>>>>
>>>>> Both ka-map and mapbuilder have both implemented much of the same
>>>>> functionality. We will need to work out a logical place to split
>>>>> between the ka-map code and the mapbuilder code.
>>>>>
>>>>> I think it would be difficult to use Mapbuilder without Mapbuilder
>>>>> controls. Probably what we would need to do is convert your ka-
>>>>> map
>>>>> tools to fit within the Mapbuilder framework.
>>>>>
>>>>> Paul Spencer wrote:
>>>>>> Cameron,
>>>>>> what do you mean by standard url? ka-Map has a javascript API
>>>>>> that
>>>>>> handles all the navigation including the smooth panning of the
>>>>>> map. While not exactly like google, the api is roughly
>>>>>> equivalent.
>>>>>> Not sure if you could use mapbuilder's controls or not, it
>>>>>> would be
>>>>>> cool if you could.
>>>>>> I'll take a stab at installing/running MapBuilder this
>>>>>> weekend. Do
>>>>>> I need to do anything special to use the Google Map layer?
>>>>>> Cheers
>>>>>> Paul
>>>>>> On 13-Apr-06, at 6:26 AM, Cameron Shorter wrote:
>>>>>>> Paul,
>>>>>>> I'm currently allowing a Google Map layer to be included as
>>>>>>> one of
>>>>>>> the layers rendered by Mapbuilder.
>>>>>>>
>>>>>>> So if you have a standard URL used to call a tiled map server,
>>>>>>> then we can create a Mapbuilder Layer which accesses the URL
>>>>>>> (we'd probably base our code on your existing ka-maps code).
>>>>>>>
>>>>>>> I'd expect that we would use the Mapbuilder pan/zoom type tools
>>>>>>> and buttons rather than the ka-map tools.
>>>>>>>
>>>>>>> Is that what you were thinking about?
>>>>>>>
>>>>>>> Paul Spencer wrote:
>>>>>>>
>>>>>>>> What would be really nice would be to have a MapBuilder widget
>>>>>>>> for ka- Map ;) Mike Adair has mentioned that this is both
>>>>>>>> possible and quite easy to do. Not being very familiar with
>>>>>>>> MapBuilder, I can't comment on that. But if it were
>>>>>>>> possible to
>>>>>>>> do, ka-Map would provide you a tiled interface and you
>>>>>>>> could use
>>>>>>>> your existing map file to do it without breaking ogc
>>>>>>>> compatibility.
>>>>>>>> Cheers
>>>>>>>> Paul
>>>>>>>> On 12-Apr-06, at 4:33 AM, Steven M. Ottens wrote:
>>>>>>>>
>>>>>>>>> Hi Bart,
>>>>>>>>>
>>>>>>>>> Thanks for the mailinglist tip.
>>>>>>>>> It might be possible to rasterize our vector data, but
>>>>>>>>> we've got
>>>>>>>>> quite
>>>>>>>>> some datasets and there will be more of them. It also makes us
>>>>>>>>> less
>>>>>>>>> flexible. Obviously we've to make a trade-off on flexibility
>>>>>>>>> and speed.
>>>>>>>>> We hope that with cached tiles we can have both the
>>>>>>>>> flexibility
>>>>>>>>> and
>>>>>>>>> speed. If we rasterise it would make sense to rasterise into
>>>>>>>>> fixed tiles
>>>>>>>>> I guess.
>>>>>>>>>
>>>>>>>>> Steven
>>>>>>>>>
>>>>>>>>> Bart van den Eijnden wrote:
>>>>>>>>>
>>>>>>>>>> Hi Steven,
>>>>>>>>>>
>>>>>>>>>> there is an OGC list to discuss tiling:
>>>>>>>>>>
>>>>>>>>>> http://lists.eogeo.org/mailman/listinfo/tiling
>>>>>>>>>>
>>>>>>>>>> check the archives there.
>>>>>>>>>>
>>>>>>>>>> What you could also do, since I assume your data is pretty
>>>>>>>>>> much static, is
>>>>>>>>>> use UMN Mapserver to rasterize all your vector data (create
>>>>>>>>>> Geotiffs e.g.)
>>>>>>>>>> and serve out the rasters instead. I guess the
>>>>>>>>>> classifications
>>>>>>>>>> used are
>>>>>>>>>> heavy on CPU usage, and pre-classified rasters would solve
>>>>>>>>>> that part at
>>>>>>>>>> least.
>>>>>>>>>>
>>>>>>>>>> Best regards,
>>>>>>>>>> Bart
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> Hi all,
>>>>>>>>>>>
>>>>>>>>>>> For those who don't know: I'm one of the lead developers on
>>>>>>>>>>> the EduGIS
>>>>>>>>>>> project, a site which aims to introduce highschool students
>>>>>>>>>>> to GIS. The
>>>>>>>>>>> site contains a webmapping part, which combined with so
>>>>>>>>>>> called
>>>>>>>>>>> 'lesson-modules' can be used inside the classroom.
>>>>>>>>>>> We aim to be able to have at least 200 students access the
>>>>>>>>>>> site at one
>>>>>>>>>>> time. Since it's being used inside the classroom it means
>>>>>>>>>>> that
>>>>>>>>>>> a lot of
>>>>>>>>>>> students will do more or less the same thing at the same
>>>>>>>>>>> time
>>>>>>>>>>> (following
>>>>>>>>>>> the tasks in the lesson-modules). This causes quite some
>>>>>>>>>>> stress on the
>>>>>>>>>>> server. Since it's a free site we have limited budget and
>>>>>>>>>>> cannot put a
>>>>>>>>>>> google-style server park behind it ;(
>>>>>>>>>>>
>>>>>>>>>>> Right now we've got apache to serverside cache the umn
>>>>>>>>>>> mapserver output.
>>>>>>>>>>> This obviously works only for those images that are
>>>>>>>>>>> requested
>>>>>>>>>>> twice
>>>>>>>>>>> (like the first mapimage). Right now about 30 students can
>>>>>>>>>>> work at the
>>>>>>>>>>> same time without it becoming unworkable slow. Since
>>>>>>>>>>> serverside caching
>>>>>>>>>>> works, the next logical step to boost performance would be
>>>>>>>>>>> using tiles
>>>>>>>>>>> like google.
>>>>>>>>>>> But..
>>>>>>>>>>> From what I know, tiling breaks OGC compatibility, right?
>>>>>>>>>>> We're using Mapbuilder as client and we prefer to keep using
>>>>>>>>>>> it, since
>>>>>>>>>>> it's turning into a rather featureful client on the EduGIS
>>>>>>>>>>> site. But the
>>>>>>>>>>> question arises how to implement tiling inside
>>>>>>>>>>> WMC/OWS-context without
>>>>>>>>>>> breaking OGC too much.
>>>>>>>>>>> Also I'm interested in the used algorithms to get tiling
>>>>>>>>>>> working.
>>>>>>>>>>>
>>>>>>>>>>> Cheers,
>>>>>>>>>>> Steven
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> ------------------------------------------------------------
>>>>>>>>>>> ------
>>>>>>>>>>>
>>>>>>>>>>> -- -
>>>>>>>>>>> To unsubscribe, e-mail:
>>>>>>>>>>> webmap-discuss-unsubscribe at mail.osgeo.org
>>>>>>>>>>> For additional commands, e-mail: webmap-discuss-
>>>>>>>>>>> help at mail.osgeo.org
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> -------------------------------------------------------------
>>>>>>>>>> ------
>>>>>>>>>>
>>>>>>>>>> --To unsubscribe, e-mail:
>>>>>>>>>> webmap-discuss-unsubscribe at mail.osgeo.org
>>>>>>>>>> For additional commands, e-mail:
>>>>>>>>>> webmap-discuss-help at mail.osgeo.org
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --------------------------------------------------------------
>>>>>>>>> ------
>>>>>>>>>
>>>>>>>>> -
>>>>>>>>> To unsubscribe, e-mail: webmap-discuss-
>>>>>>>>> unsubscribe at mail.osgeo.org
>>>>>>>>> For additional commands, e-mail:
>>>>>>>>> webmap-discuss-help at mail.osgeo.org
>>>>>>>>>
>>>>>>>> +--------------------------------------------------------------
>>>>>>>> ---+
>>>>>>>> |Paul Spencer
>>>>>>>> pspencer at dmsolutions.ca |
>>>>>>>> +--------------------------------------------------------------
>>>>>>>> ---+
>>>>>>>> |Applications & Software
>>>>>>>> Development |
>>>>>>>> |DM Solutions Group Inc http://
>>>>>>>> www.dmsolutions.ca/|
>>>>>>>> +--------------------------------------------------------------
>>>>>>>> ---+
>>>>>>>> ---------------------------------------------------------------
>>>>>>>> ------
>>>>>>>>
>>>>>>>> To unsubscribe, e-mail: webmap-discuss-
>>>>>>>> unsubscribe at mail.osgeo.org
>>>>>>>> For additional commands, e-mail: webmap-discuss-
>>>>>>>> help at mail.osgeo.org
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --Cameron Shorter
>>>>>>> http://cameron.shorter.net
>>>>>>>
>>>>>>> ----------------------------------------------------------------
>>>>>>> -----
>>>>>>>
>>>>>>> To unsubscribe, e-mail: webmap-discuss-
>>>>>>> unsubscribe at mail.osgeo.org
>>>>>>> For additional commands, e-mail: webmap-discuss-
>>>>>>> help at mail.osgeo.org
>>>>>>>
>>>>>> +----------------------------------------------------------------
>>>>>> -+
>>>>>> |Paul Spencer
>>>>>> pspencer at dmsolutions.ca |
>>>>>> +----------------------------------------------------------------
>>>>>> -+
>>>>>> |Applications & Software
>>>>>> Development |
>>>>>> |DM Solutions Group Inc http://
>>>>>> www.dmsolutions.ca/|
>>>>>> +----------------------------------------------------------------
>>>>>> -+
>>>>>> -----------------------------------------------------------------
>>>>>> ----
>>>>>> To unsubscribe, e-mail: webmap-discuss-unsubscribe at mail.osgeo.org
>>>>>> For additional commands, e-mail: webmap-discuss-
>>>>>> help at mail.osgeo.org
>>>>>
>>>>>
>>>>> --Cameron Shorter
>>>>> http://cameron.shorter.net
>>>>>
>>>>> ------------------------------------------------------------------
>>>>> ---
>>>>> To unsubscribe, e-mail: webmap-discuss-unsubscribe at mail.osgeo.org
>>>>> For additional commands, e-mail: webmap-discuss-
>>>>> help at mail.osgeo.org
>>>>>
>>>>
>>>> +-----------------------------------------------------------------+
>>>> |Paul Spencer pspencer at dmsolutions.ca |
>>>> +-----------------------------------------------------------------+
>>>> |Applications & Software Development |
>>>> |DM Solutions Group Inc http://www.dmsolutions.ca/|
>>>> +-----------------------------------------------------------------+
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> -------------------------------------------------------------------
>>>> --
>>>> To unsubscribe, e-mail: webmap-discuss-unsubscribe at mail.osgeo.org
>>>> For additional commands, e-mail: webmap-discuss-help at mail.osgeo.org
>>>>
>>>
>>>
>>>
>>> --------------------------------------------------------------------
>>> -
>>> To unsubscribe, e-mail: webmap-discuss-unsubscribe at mail.osgeo.org
>>> For additional commands, e-mail: webmap-discuss-help at mail.osgeo.org
>>>
>>
>> +-----------------------------------------------------------------+
>> |Paul Spencer pspencer at dmsolutions.ca |
>> +-----------------------------------------------------------------+
>> |Applications & Software Development |
>> |DM Solutions Group Inc http://www.dmsolutions.ca/|
>> +-----------------------------------------------------------------+
>>
>>
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: webmap-discuss-unsubscribe at mail.osgeo.org
>> For additional commands, e-mail: webmap-discuss-help at mail.osgeo.org
>>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: webmap-discuss-unsubscribe at mail.osgeo.org
> For additional commands, e-mail: webmap-discuss-help at mail.osgeo.org
>
+-----------------------------------------------------------------+
|Paul Spencer pspencer at dmsolutions.ca |
+-----------------------------------------------------------------+
|Applications & Software Development |
|DM Solutions Group Inc http://www.dmsolutions.ca/|
+-----------------------------------------------------------------+
More information about the Mail_webmap-discuss
mailing list