[gdal-dev] off-topic: Canadian geospatial data, updating

[Matt Wilkie]

Hello fellow canadian spatial data users,

As many are aware, though perhaps some are not so I'll provide some 
background, Natural Resources Canada is releasing 1:50,000 or better 
scale topographic data on an ongoing 6 month release cycle under the 
name Canvec 
<http://geogratis.cgdi.gc.ca/geogratis/en/collection/5460AA9D-54CD-8349-C95E-1A4D03172FDF.html;jsessionid=06CE8DE7479E1DF0AB8D2E3226176469>. 
It is available piecemeal by individual National Topographic Series 50k 
tiles, grouped by province, and as a single national dataset. The 
all-in-one package is 21.5gb as file geodatabase or 27gb as GML 
<ftp://ftp2.cits.rncan.gc.ca/pub/canvec/canada_gml/>. This is great news!

Acquiring the data is not as smooth as it could be. The ftp server 
limits 2 concurrent sessions from the same ip and throttles the 
connections as well. Http downloads are faster and not limited 
concurrently; the national dataset is available by ftp only. So one can 
have fast downloads and grab a lot at once via http, but that requires 
installing and wrestling with a bulk download manager. Or one can use an 
ftp client recursively, and go slow (and not be able to browse the site 
while downloading, only 2 connections per ip remember). Thank heavens 
resume download is supported (but be careful with wget 
<http://code.google.com/p/maphew/issues/detail?id=14>).

This is pain worth going through for a dataset this valuable, but 
remember it happens at least twice year. The spring release is usually 
April, however this year there is a follow up bug-fix (discrepancies in 
their parlance) release expected in the next month or so.

I've written them twice in the last couple of years to ask them to make 
the data available by rsync as well so that only the changes need be 
downloaded, resulting in a faster update as well as tremendous bandwidth 
savings. For the first request I received acknowledgment in the form of 
"that's interesting, we'll look into it" and for the second, silence. I 
conclude there will be little if any movement on that front.

So now we come to why I am writing to all of you: are you aware of any 
data warehouse service which might be coaxed into mirroring this data 
and making it available by rysnc?

thank you for your thoughts,

-- 
matt wilkie
--------------------------------------------
Geomatics Analyst
Information Management and Technology
Yukon Department of Environment
10 Burns Road * Whitehorse, Yukon * Y1A 4Y9
867-667-8133 Tel * 867-393-7003 Fax
http://environmentyukon.gov.yk.ca/geomatics/
--------------------------------------------

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.osgeo.org/pipermail/gdal-dev/attachments/20100521/1298bbd3/attachment.html