<div dir="ltr">Hi, <br><br>here it is:<br><br><blockquote style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;" class="gmail_quote"><sigq> Title: Skygone Cloud, Cloud GIS, SaaS - Skygone Cloud, Private Cloud, SaaS (at <a href="http://www.skygoneinc.com">www.skygoneinc.com</a>)<br>
<lucdoneaERDAS> @jmckenna: I see on the wiki: "WMS output formats to be used will be png8 and png24 where possible". Didn't we agrred that jpeg output would be use on imagery data sources and png 8/24 on vector data sources?<br>
<acuster> (if we are back to the earlier topic, didn't we agree that rule 1 was not a rule but a principle?)<br><jmckenna> lucdoneaERDAS: you could be right, i will check the logs. can anyone else confirm this jpeg/imagery plan?<br>
* springmeyer Dane Springmeyer arrives - Mapnik<br><mpdaly> jmckenna: VMs are no good because each test run could have different host resources, I guess<br><jmckenna> mpdaly: yes agreed, resource use could fluctuate<br>
<msmith_> we are in the midst of getting new servers but I don't think we'll have any newer ones available in time. We do have the servers we used last year but they are getting a bit long in the tooth.<br><jmckenna> msmith_: Did you find out anything on your end regarding hosting the machines for this year?<br>
<ascollignon> jmckenna: regarding the jpeg output, it was in the last year rules, but this was not clearly discussed during the last meeting, we were more focused on the vector output, not so much about the raster ouput. It should be nice to clarify it. I would be +1 to have jpeg as raster ouput.<br>
<-- lucdoneaERDAS (~<a href="mailto:ldonea@mail.ionicsoft.com">ldonea@mail.ionicsoft.com</a>) has left #foss4g<br><msmith_> I think we will have a new public T1 line so the network throughput should be much better than our old DSL line<br>
--> lucdoneaERDAS (~<a href="mailto:ldonea@mail.ionicsoft.com">ldonea@mail.ionicsoft.com</a>) has joined #foss4g<br><jmckenna> that's great news. the alternative is that we start looking at a paid host for a few months. but i'd prefer to use msmith_ as he/they hosted the machines well for last year's exercise<br>
<msmith_> except for network speed :)<br><jmckenna> true ha.<br><jmckenna> msmith_: any problems getting 3 dedicated machines, with one windows ?<br><msmith_> For having windows and linux, should I set up 2 VMs on the testing machine, one for each OS and then we can start/stop each one for testing?<br>
<msmith_> Are we going to set it up differently that last time? We had 3 boxes, tester, wms server and backend database on 3 separate machines<br><jmckenna> it would be nice to have 3 machines: one machine each for linux and windows WMS Servers, then another machine to host the tests. then maybe a 4th machine to host databases. <br>
<jmckenna> :) am i asking too much?<br><jmckenna> 5 minutes left for server discussions. anyone have more thoughts/needs for servers, let msmith_ know, as they (US Army Corps of Engineers) may be the ones hosting our test machines.<br>
<jmckenna> so speak now :)<br><lucdoneaERDAS> It would be nice to have a 64bits server for Windows<br><msmith_> We don't have multiple identical machines so the linux and windows machines wouldn't be comparable<br>
<msmith_> The specs for the machines are on the wiki. I don't know if 64bit would make a difference, but we can try<br><IvanSanchez> this means the Linux machines are also 64bit, right?<br><IvanSanchez> i.e. running a 64bit kernel<br>
--> simboss (~cda6af64@gateway/web/freenode/ip.205.166.175.100) has joined #foss4g<br><lucdoneaERDAS> msmith_ can you provide a link to these sepecs?<br><lucdoneaERDAS> Also we would need to have Windows Server 2003 or Windows Server 2008 as OS<br>
<jmckenna> lucdoneaERDAS: your windows requests look reasonable. <br><acuster> win 2003 blocks a 3.6 Gb, no?<br><mpdaly> acuster: FUD<br><lucdoneaERDAS> acuster what do you mean?<br><mpdaly> <a href="http://msdn.microsoft.com/en-us/library/aa366778(VS.85).aspx#physical_memory_limits_windows_server_2003">http://msdn.microsoft.com/en-us/library/aa366778(VS.85).aspx#physical_memory_limits_windows_server_2003</a><br>
<sigq> Title: Unable to service request (at <a href="http://msdn.microsoft.com">msdn.microsoft.com</a>)<br><mpdaly> 32-bit up to 128GB, 64-bit up to 2 TB<br><lucdoneaERDAS> Windows Server 2008 and 64bits are ok for us<br>
--> pramsey (~<a href="mailto:pramsey@topp148-office-nyc.openplans.org">pramsey@topp148-office-nyc.openplans.org</a>) has joined #foss4g<br><acuster> cool, I was mis-informed<br><msmith_> here's the specs link <a href="http://wiki.osgeo.org/wiki/Benchmarking_2009#Hardware">http://wiki.osgeo.org/wiki/Benchmarking_2009#Hardware</a><br>
<sigq> Title: Benchmarking 2009 - OSGeo Wiki (at <a href="http://wiki.osgeo.org">wiki.osgeo.org</a>)<br><acuster> or not, seems it depends on the kind of win 2003<br><mpdaly> I imagine that USACE has access to a variety<br>
--> pramsey_ (~<a href="mailto:pramsey@topp148-office-nyc.openplans.org">pramsey@topp148-office-nyc.openplans.org</a>) has joined #foss4g<br><-- pramsey has quit (Read error: No route to host)<br>* pramsey_ is now known as pramsey<br>
<jmckenna> ok we will move those specs to new wiki, and record the desired OS specs there also. can we move onto dataset now?<br><jmckenna> IvanSanchez: your lead :)<br><IvanSanchez> jmckenna: thanks<br>
<IvanSanchez> Earlier this week I set up this wiki page: <a href="http://wiki.osgeo.org/wiki/Benchmarking_2010/How_to_get_some_sample_data">http://wiki.osgeo.org/wiki/Benchmarking_2010/How_to_get_some_sample_data</a><br>
<sigq> Title: Benchmarking 2010/How to get some sample data - OSGeo Wiki (at <a href="http://wiki.osgeo.org">wiki.osgeo.org</a>)<br><acuster> msmith_, the first machine has how much ram? 4x2GB?<br><IvanSanchez> it details how to download some data, and how it will look<br>
--> FrankW (~<a href="mailto:FrankW@ip-174-142-75-109.static.privatedns.com">FrankW@ip-174-142-75-109.static.privatedns.com</a>) has joined #foss4g<br>--> trevorw (~<a href="mailto:trevor_we@96.51.206.244">trevor_we@96.51.206.244</a>) has joined #foss4g<br>
<msmith_> correct. 8gb<br><mpdaly> IvanSanchez: I found the download quite unreliable, but got there in the end (for the two BCN sheets)<br><IvanSanchez> What I've been doing is downloading the full set of vector data (about 400.000 shapefiles!), and merging them<br>
<IvanSanchez> mpdaly: complain at the spanish mapping agency :-)<br><jmckenna> IvanSanchez: I did download one same file, zone 420 for barcelona i think. lots of files!<br><mpdaly> Happily!<br><jmckenna> same/sample<br>
<IvanSanchez> OK, so if you downloaded a sheet of vector data and unzipped it, you found about 100 different shapefiles<br><acuster> IvanSanchez, why transform them to WGS84?<br><-- pirmin_k has quit (Remote host closed the connection)<br>
<mpdaly> acuster: beat me to it<br><IvanSanchez> acuster: just so they share the same SRS, right now they are in four different SRSs<br><jmckenna> last year for vector data we had one file for lines (roads), one file for polygons (state land), and one file for points (cities)<br>
<IvanSanchez> my current goal is to provide a set of shapefiles, one per theme, e.g.:<br><IvanSanchez> one shapefile for admin boundaries<br><IvanSanchez> one shapefile for roads<br><IvanSanchez> one shapefile for motorways<br>
<IvanSanchez> etc<br><jmckenna> we need one poly, one point, one line. that's all<br><IvanSanchez> jmckenna: unfortunately one poly, one line and one point will not be feasible<br><IvanSanchez> I'm already having problems with shapefiles > 2 GB<br>
<IvanSanchez> the realistic thing to do is one shapefile per theme<br><IvanSanchez> Otherwise I'll hit the 2 GB limit per shapefile<br><jmckenna> IvanSanchez: well we cannot style a pretty map from 100 shapefiles. the work todo is agreeing on 3 files and then processing the data for those 3 files<br>
<jmckenna> if that cannot be done then lets use last year's data<br><IvanSanchez> I'm thinking of styling no more than 6 or 10 shapefiles<br><IvanSanchez> probably the ones with most data<br><jmckenna> i'm not closing the door, but that's the reality (time is limited, for data processing work)<br>
<jmckenna> one point, one poly, one line. where do you get 6 or 10?<br><jmckenna> are you thinking 'pretty map' ?<br><-- igrcic (~<a href="mailto:chatzilla@178.160.48.100">chatzilla@178.160.48.100</a>) has left #foss4g<br>
<IvanSanchez> I'm thinking "half decent"<br><IvanSanchez> if you see the table at <a href="http://wiki.osgeo.org/wiki/Benchmarking_2010/How_to_get_some_sample_data">http://wiki.osgeo.org/wiki/Benchmarking_2010/How_to_get_some_sample_data</a> , I've bolded a few themes<br>
<sigq> Title: Benchmarking 2010/How to get some sample data - OSGeo Wiki (at <a href="http://wiki.osgeo.org">wiki.osgeo.org</a>)<br><IvanSanchez> the bold themes are the "most interesting" ones, and IMHO should be the only ones used<br>
<jmckenna> the test is rendering, speed. we're not publishing these maps. <br><IvanSanchez> I do agree there's no point in styling mills<br><acuster> but pretty maps are sexy<br><IvanSanchez> I do agree time is limited, but IMHO the tests should cover per-attribute styling<br>
<IvanSanchez> (i.e. colouring roads differently depending on the type of road)<br><jmckenna> we cannot ask teams to do this<br><IvanSanchez> I'd like to hear opinions from the teams at this point :-)<br>
<IvanSanchez> how much styling are you guys willing to do?<br><acuster> yeah, seems reasonable<br><jmckenna> were you involved in last year's event? did you see how much work it involved?<br><jmckenna> i'm sorry<br>
<IvanSanchez> (no, I wans't involved in last years')<br><jmckenna> that's my opinion, having worked a month on it with aaime last year<br><acuster> it depends if the benchmark should evaluate access time to data or work time of the WMS engine<br>
<jmckenna> exactly my point<br><acuster> since it's a WMS benchmark, I'd be inclined towards the latter<br><jmckenna> we must limit the data to a minimum. you can have lots of coverage, but we're all going to be transfering the data from our local machine etc. i guess you'll see soon what i mean<br>
<IvanSanchez> jmckenna: I have a general idea<br><IvanSanchez> anyway, I will have to provide more than just 3 shapefiles, if just for the fact that the shapefile format has a hard limit of 2 GB in size<br><IvanSanchez> I will also provide a torrent (with a 500 KBs/sec seeder) for the data<br>
<jmckenna> that's a great way to share the data. good plan.<br><IvanSanchez> I will be working on putting the data together this week. In the meantime, I encourage teams to download some vector data, examine it, and see how many layers/themes they would be willing/wanting to style<br>
<IvanSanchez> Shall I move to the raster data bit?<br><jmckenna> *sigh it's not want, it's regarding what we need to test. <br><IvanSanchez> Hey, you guys reach a consensus about what you need to test. I'm here just to give you data :-)<br>
<jmckenna> my thoughts exactly, why are we arguing? lol<br><IvanSanchez> moving onto the raster data...<br><acuster> IvanSanchez, a question on your styling rule, #rendering notes, it seems you are outside the limits of OGC sld<br>
<IvanSanchez> acuster: that's part of the fun<br><jmckenna> 5 minutes left<br><acuster> ah, well for our part we are working with the OGC on sld-next<br><IvanSanchez> *personally* I'd like to see what every rendering engine is capable of doing<br>
<IvanSanchez> but that's just my personal point of view<br><acuster> we're capable, but its' non-standard<br><mpdaly> IvanSanchez: That's an interesting test, no doubt, but a different one<br>
<IvanSanchez> acuster: the winner of my personal show-me-your-best-rendering wins a free beer<br><jmckenna> worst case scenario we use last year's data. it seems IvanSanchez wants to provide data with the catch that we must change the tests<br>
<IvanSanchez> jmckenna: I'll try to keep things simple<br><ascollignon> where/when do we stand that SLD was a rules for the rendering?<br><jmckenna> one of the teams asked if SLD is a requirement for this exercise. in fact it is not. hence why we must limit the rendering/style to bare minimum.<br>
<jmckenna> (one team does not support SLD)<br><IvanSanchez> about raster data: I'll be getting a hard drive full or orthophotos, convert them to geotiff, put them into a torrent. Stay tuned.<br><IvanSanchez> jmckenna: is roads with casing asking too much?<br>
<acuster> seems an sld test would be a resonable thing to add, and one team doesn't play in that test<br><jmckenna> IvanSanchez: example styling from last year (see map image below): <a href="http://wiki.osgeo.org/wiki/Texas_roads_styled">http://wiki.osgeo.org/wiki/Texas_roads_styled</a><br>
<sigq> Title: Texas roads styled - OSGeo Wiki (at <a href="http://wiki.osgeo.org">wiki.osgeo.org</a>)<br><acuster> IvanSanchez, these will already be a mosaic?<br><IvanSanchez> acuster: the raster data will be something *about* 50 files, each one covering several square kilometers<br>
<IvanSanchez> each file is a processed mosaic, if that's what you mean<br><acuster> I'm wondering if they are aligned<br><acuster> or if they overlap<br><IvanSanchez> acuster: I *think* they overlap just a little bit<br>
<acuster> thanks<br><IvanSanchez> acuster: I will be putting a schematic of how much raster data I'll be trying to get<br><IvanSanchez> I think they overlap a little bit in the borders, if just to prevent gaps when rendering<br>
<IvanSanchez> acuster: see <a href="http://wiki.osgeo.org/wiki/Benchmarking_2010/How_to_get_some_sample_data#Raster_data_issues">http://wiki.osgeo.org/wiki/Benchmarking_2010/How_to_get_some_sample_data#Raster_data_issues</a><br>
<sigq> Title: Benchmarking 2010/How to get some sample data - OSGeo Wiki (at <a href="http://wiki.osgeo.org">wiki.osgeo.org</a>)<br><IvanSanchez> I've uploaded an image of just how much raster data, and how it will be distributed<br>
<IvanSanchez> basically, one file per square in that diagram<br><IvanSanchez> I will provide size estimates (in GB) for the raster data as soon as I start converting ECWs into geotiffs<br><jmckenna> we have reached our meeting time limit of one hour. I'd like to see feedback on data from all teams sent to the email list, so IvanSanchez knows what layers to pre-process. If the teams all want pretty maps then we can do pretty maps this year, it depends on what we agree to. Next week's meeting: I will be attending an event (with danmo and msmith_) so I cannot attend. i will follow logs...<br>
<jmckenna> ...and meeting summary.<br><ascollignon> IvanSanchez, as for the best effort ECW will be also use, could you provide both ECW and Geotiff? In order to avoid to re-convert internaly?<br><acuster> IvanSanchez, thanks for the work<br>
<IvanSanchez> ascollignon: I can provide both datasets, will probably set up a torrent for each<br><acuster> IvanSanchez, are you going to reproject the GeoTiff's also?<br><IvanSanchez> acuster: you're welcome<br>
<acuster> if not, what are they in?<br><IvanSanchez> acuster: I don't plan to reproject geotiffs - my single-core at home should not cope with that<br><acuster> ok<br><IvanSanchez> acuster: download one ECW image (see wiki instructions) and see for yourself<br>
<IvanSanchez> I cannot recall<br>* IvanSanchez shuts up<br><acuster> ecw is unfortunately a terrifically encumbered technology<br><acuster> I'll wait for your geotiffs<br><-- marco___ has quit (Remote host closed the connection)<br>
<msmith_> I won't be on next weeks meeting as jmckenna said, but it would be good to decide on server setup so I can get the machines available on the net if we are going to use the ones from last year. <br><acuster> msmith_, seems you are the only one with hardware to offer so we will probably accept gladly<br>
<msmith_> I think the only option for the wms server will be as VMs as we will want to run each OS on an identical box and this is the only way I can think of without swapping drives<br><IvanSanchez> as jmckenna said, I'd like to see feedback from the teams to see how pretty do we want our maps<br>
<acuster> IvanSanchez, that goes along with my desire to know what we will be testing<br><acuster> reproject/not, sld/not, ...<br><springmeyer> IvanSanchez: I think for styles its going to take someone proposing a specific SLD, like andrea did last year, that can be exactly matched<br>
<springmeyer> otherwise is must be very very simple to keep consistency<br><msmith_> can we use the same SLD on the new data?<br><IvanSanchez> acuster: I do agree. Personally, I'd like to see how WMS servers perform on rendering semi-complex simbology: flowing antialiased text, cased roads, dashed railways...<br>
<acuster> the sld on line looks iffy<br><IvanSanchez> msmith_: no, if just because the fields names of the shapefiles differ from last year's<br><IvanSanchez> msmith_: they could be tweaked, though<br>
<acuster> IvanSanchez, the problem with that is that if it's beyond the spec, no clients can know how to ask for it<br><msmith_> I meant the symbology. The fields can change<br><IvanSanchez> msmith_: Maybe the SLDs will have to contemplate an extra color for the roads or so, but it's doable<br>
<msmith_> acuster: that can be part of the "best foot" display. Here's the base SLD and your "enhanced" non-standard SLD<br><acuster> msmith_, but then what are we doing if one has beautiful maps a little bit slower than another who has pretty maps?<br>
<msmith_> Is anybody going to need SDE support or is just postgres and oracle db support sufficient?<br><acuster> hard to compare<br><IvanSanchez> acuster: it's hard to compare *anyway*<br><FrankW> to the best of my knowledge we aren't trying to do any SDE this year.<br>
<msmith_> acuster: thats why we have 2 categories. Base and Best<br><msmith_> FrankWL one less thing for me to configure and install<br><IvanSanchez> yeah. and the best Best wins a beer from me.<br><FrankW> I'm sort of interested in also setting up an ingres instance with spatial support though it might only be to compare mapserver ingres/oracle/postgis performance.<br>
<acuster> so each team would have to decide whether to aim for pretty or for fast?<br><msmith_> If you can provide the install media, I can install it. OS requirements?<br><acuster> oh, for beer we'll definately compete for the pretty then<br>
<acuster> :-)<br><IvanSanchez> acuster: first, aim for fast (base). When you're done, aim for pretty (best)<br><msmith_> FrankW: I'd be curious about that. <br><IvanSanchez> basically, base will be shown on the big screens, while the beer is... well, having a beeer in a pub filled with geeks<br>
<acuster> the 'base' as it is being talked about seems strange<br><acuster> for instance, are we allowed to pyramid the geotiffs?<br><FrankW> it was agreed that pyramid would be built for the base geotiffs.<br>
<acuster> ah, we have to use a common pyramid?<br><FrankW> not necessarily, no.<br><-- ascollignon (~<a href="mailto:ascollign@mail.ionicsoft.com">ascollign@mail.ionicsoft.com</a>) has left #foss4g<br><FrankW> The <a href="http://logs.qgis.org/foss4g/%23foss4g.2010-06-09.log">http://logs.qgis.org/foss4g/%23foss4g.2010-06-09.log</a> isn't up to date yet, was there any resolution on where we will get machines to run the shootout?<br>
<sigq> Title: IRC Log - #FOSS4G (at <a href="http://logs.qgis.org">logs.qgis.org</a>)<br><acuster> FrankW, msmith_ is the only one with an offer on the table right now, from what I understand<br><acuster> and has upgraded to a T1 line from DSL, if I understood correctly<br>
<FrankW> Thanks, I did read that part. <br><FrankW> I'm a bit nervous about having to run the main test systems in a VM. <br><FrankW> And switching VMs will mean even more contention.<br><FrankW> But perhaps we just have to live with that.<br>
<msmith_> I'm open to ideas but we only have so many machines available<br><msmith_> And I've got to run. I'll check logs for more.<br><FrankW> yes, of course I understand!<br><-- msmith_ has quit (Quit: Page closed)<br>
</blockquote></div>