[Benchmarking] Whooops....

[mohamed khalloufi]

Hi,

here it is:

<sigq> Title: Skygone Cloud, Cloud GIS, SaaS - Skygone Cloud, Private Cloud,
> SaaS (at www.skygoneinc.com)
> <lucdoneaERDAS> @jmckenna: I see on the wiki: "WMS output formats to be
> used will be png8 and png24 where possible". Didn't we agrred that jpeg
> output would be use on imagery data sources and png 8/24 on vector data
> sources?
> <acuster> (if we are back to the earlier topic, didn't we agree that rule 1
> was not a rule but a principle?)
> <jmckenna> lucdoneaERDAS: you could be right, i will check the logs.  can
> anyone else confirm this jpeg/imagery plan?
> * springmeyer Dane Springmeyer arrives - Mapnik
> <mpdaly> jmckenna: VMs are no good because each test run could have
> different host resources, I guess
> <jmckenna> mpdaly: yes agreed, resource use could fluctuate
> <msmith_> we are in the midst of getting new servers but I don't think
> we'll have any newer ones available in time. We do have the servers we used
> last year but they are getting a bit long in the tooth.
> <jmckenna> msmith_: Did you find out anything on your end regarding hosting
> the machines for this year?
> <ascollignon> jmckenna: regarding the jpeg output, it was in the last year
> rules, but this was not clearly discussed during the last meeting, we were
> more focused on the vector output, not so much about the raster ouput. It
> should be nice to clarify it. I would be +1 to have jpeg as raster ouput.
> <-- lucdoneaERDAS (~ldonea at mail.ionicsoft.com) has left #foss4g
> <msmith_> I think we will have a new public T1 line so the network
> throughput should be much better than our old DSL line
> --> lucdoneaERDAS (~ldonea at mail.ionicsoft.com) has joined #foss4g
> <jmckenna> that's great news.  the alternative is that we start looking at
> a paid host for a few months.  but i'd prefer to use msmith_ as he/they
> hosted the machines well for last year's exercise
> <msmith_> except for network speed :)
> <jmckenna> true ha.
> <jmckenna> msmith_: any problems getting 3 dedicated machines, with one
> windows ?
> <msmith_> For having windows and linux, should I set up 2 VMs on the
> testing machine, one for each OS and then we can start/stop each one for
> testing?
> <msmith_> Are we going to set it up differently that last time? We had 3
> boxes, tester, wms server and backend database on 3 separate machines
> <jmckenna> it would be nice to have 3 machines: one machine each for linux
> and windows WMS Servers, then another machine to host the tests.  then maybe
> a 4th machine to host databases.
> <jmckenna> :) am i asking too much?
> <jmckenna> 5 minutes left for server discussions.  anyone have more
> thoughts/needs for servers, let msmith_ know, as they (US Army Corps of
> Engineers) may be the ones hosting our test machines.
> <jmckenna> so speak now :)
> <lucdoneaERDAS> It would be nice to have a 64bits server for Windows
> <msmith_> We don't have multiple identical machines so the linux and
> windows machines wouldn't be comparable
> <msmith_> The specs for the machines are on the wiki. I don't know if 64bit
> would make a difference, but we can try
> <IvanSanchez> this means the Linux machines are also 64bit, right?
> <IvanSanchez> i.e. running a 64bit kernel
> --> simboss (~cda6af64 at gateway/web/freenode/ip.205.166.175.100) has joined
> #foss4g
> <lucdoneaERDAS> msmith_ can you provide a link to these sepecs?
> <lucdoneaERDAS> Also we would need to have Windows Server 2003 or Windows
> Server 2008 as OS
> <jmckenna> lucdoneaERDAS: your windows requests look reasonable.
> <acuster> win 2003 blocks a 3.6 Gb, no?
> <mpdaly> acuster: FUD
> <lucdoneaERDAS> acuster what do you mean?
> <mpdaly>
> http://msdn.microsoft.com/en-us/library/aa366778(VS.85).aspx#physical_memory_limits_windows_server_2003
> <sigq> Title: Unable to service request (at msdn.microsoft.com)
> <mpdaly> 32-bit up to 128GB, 64-bit up to 2 TB
> <lucdoneaERDAS> Windows Server 2008 and 64bits are ok for us
> --> pramsey (~pramsey at topp148-office-nyc.openplans.org) has joined #foss4g
> <acuster> cool, I was mis-informed
> <msmith_> here's the specs link
> http://wiki.osgeo.org/wiki/Benchmarking_2009#Hardware
> <sigq> Title: Benchmarking 2009 - OSGeo Wiki (at wiki.osgeo.org)
> <acuster> or not, seems it depends on the kind of win 2003
> <mpdaly> I imagine that USACE has access to a variety
> --> pramsey_ (~pramsey at topp148-office-nyc.openplans.org) has joined
> #foss4g
> <-- pramsey has quit (Read error: No route to host)
> * pramsey_ is now known as pramsey
> <jmckenna> ok we will move those specs to new wiki, and record the desired
> OS specs there also.  can we move onto dataset now?
> <jmckenna> IvanSanchez: your lead :)
> <IvanSanchez> jmckenna: thanks
> <IvanSanchez> Earlier this week I set up this wiki page:
> http://wiki.osgeo.org/wiki/Benchmarking_2010/How_to_get_some_sample_data
> <sigq> Title: Benchmarking 2010/How to get some sample data - OSGeo Wiki
> (at wiki.osgeo.org)
> <acuster> msmith_, the first machine has how much ram? 4x2GB?
> <IvanSanchez> it details how to download some data, and how it will look
> --> FrankW (~FrankW at ip-174-142-75-109.static.privatedns.com) has joined
> #foss4g
> --> trevorw (~trevor_we at 96.51.206.244) has joined #foss4g
> <msmith_> correct. 8gb
> <mpdaly> IvanSanchez: I found the download quite unreliable, but got there
> in the end (for the two BCN sheets)
> <IvanSanchez> What I've been doing is downloading the full set of vector
> data (about 400.000 shapefiles!), and merging them
> <IvanSanchez> mpdaly: complain at the spanish mapping agency :-)
> <jmckenna> IvanSanchez: I did download one same file, zone 420 for
> barcelona i think.   lots of files!
> <mpdaly> Happily!
> <jmckenna> same/sample
> <IvanSanchez> OK, so if you downloaded a sheet of vector data and unzipped
> it, you found about 100 different shapefiles
> <acuster> IvanSanchez, why transform them to WGS84?
> <-- pirmin_k has quit (Remote host closed the connection)
> <mpdaly> acuster: beat me to it
> <IvanSanchez> acuster: just so they share the same SRS, right now they are
> in four different SRSs
> <jmckenna> last year for vector data we had one file for lines (roads), one
> file for polygons (state land), and one file for points (cities)
> <IvanSanchez> my current goal is to provide a set of shapefiles, one per
> theme, e.g.:
> <IvanSanchez> one shapefile for admin boundaries
> <IvanSanchez> one shapefile for roads
> <IvanSanchez> one shapefile for motorways
> <IvanSanchez> etc
> <jmckenna> we need one poly, one point, one line.  that's all
> <IvanSanchez> jmckenna: unfortunately one poly, one line and one point will
> not be feasible
> <IvanSanchez> I'm already having problems with shapefiles > 2 GB
> <IvanSanchez> the realistic thing to do is one shapefile per theme
> <IvanSanchez> Otherwise I'll hit the 2 GB limit per shapefile
> <jmckenna> IvanSanchez: well we cannot style a pretty map from 100
> shapefiles.  the work todo is agreeing on 3 files and then processing the
> data for those 3 files
> <jmckenna> if that cannot be done then lets use last year's data
> <IvanSanchez> I'm thinking of styling no more than 6 or 10 shapefiles
> <IvanSanchez> probably the ones with most data
> <jmckenna> i'm not closing the door, but that's the reality (time is
> limited, for data processing work)
> <jmckenna> one point, one poly, one line.  where do you get 6 or 10?
> <jmckenna> are you thinking 'pretty map' ?
> <-- igrcic (~chatzilla at 178.160.48.100) has left #foss4g
> <IvanSanchez> I'm thinking "half decent"
> <IvanSanchez> if you see the table at
> http://wiki.osgeo.org/wiki/Benchmarking_2010/How_to_get_some_sample_data ,
> I've bolded a few themes
> <sigq> Title: Benchmarking 2010/How to get some sample data - OSGeo Wiki
> (at wiki.osgeo.org)
> <IvanSanchez> the bold themes are the "most interesting" ones, and IMHO
> should be the only ones used
> <jmckenna> the test is rendering, speed.  we're not publishing these maps.
>
> <IvanSanchez> I do agree there's no point in styling mills
> <acuster> but pretty maps are sexy
> <IvanSanchez> I do agree time is limited, but IMHO the tests should cover
> per-attribute styling
> <IvanSanchez> (i.e. colouring roads differently depending on the type of
> road)
> <jmckenna> we cannot ask teams to do this
> <IvanSanchez> I'd like to hear opinions from the teams at this point :-)
> <IvanSanchez> how much styling are you guys willing to do?
> <acuster> yeah, seems reasonable
> <jmckenna> were you involved in last year's event? did you see how much
> work it involved?
> <jmckenna> i'm sorry
> <IvanSanchez> (no, I wans't involved in last years')
> <jmckenna> that's my opinion, having worked a month on it with aaime last
> year
> <acuster> it depends if the benchmark should evaluate access time to data
> or work time of the WMS engine
> <jmckenna> exactly my point
> <acuster> since it's a WMS benchmark, I'd be inclined towards the latter
> <jmckenna> we must limit the data to a minimum.  you can have lots of
> coverage, but we're all going to be transfering the data from our local
> machine etc.  i guess you'll see soon what i mean
> <IvanSanchez> jmckenna: I have a general idea
> <IvanSanchez> anyway, I will have to provide more than just 3 shapefiles,
> if just for the fact that the shapefile format has a hard limit of 2 GB in
> size
> <IvanSanchez> I will also provide a torrent (with a 500 KBs/sec seeder) for
> the data
> <jmckenna> that's a great way to share the data.  good plan.
> <IvanSanchez> I will be working on putting the data together this week. In
> the meantime, I encourage teams to download some vector data, examine it,
> and see how many layers/themes they would be willing/wanting to style
> <IvanSanchez> Shall I move to the raster data bit?
> <jmckenna> *sigh  it's not want, it's regarding what we need to test.
> <IvanSanchez> Hey, you guys reach a consensus about what you need to test.
> I'm here just to give you data :-)
> <jmckenna> my thoughts exactly, why are we arguing? lol
> <IvanSanchez> moving onto the raster data...
> <acuster> IvanSanchez, a question on your styling rule, #rendering notes,
> it seems you are outside the limits of OGC sld
> <IvanSanchez> acuster: that's part of the fun
> <jmckenna> 5 minutes left
> <acuster> ah, well for our part we are working with the OGC on sld-next
> <IvanSanchez> *personally* I'd like to see what every rendering engine is
> capable of doing
> <IvanSanchez> but that's just my personal point of view
> <acuster> we're capable, but its' non-standard
> <mpdaly> IvanSanchez: That's an interesting test, no doubt, but a different
> one
> <IvanSanchez> acuster: the winner of my personal
> show-me-your-best-rendering wins a free beer
> <jmckenna> worst case scenario we use last year's data.  it seems
> IvanSanchez wants to provide data with the catch that we must change the
> tests
> <IvanSanchez> jmckenna: I'll try to keep things simple
> <ascollignon> where/when do we stand that SLD was a rules for the
> rendering?
> <jmckenna> one of the teams asked if SLD is a requirement for this
> exercise.  in fact it is not.  hence why we must limit the rendering/style
> to bare minimum.
> <jmckenna> (one team does not support SLD)
> <IvanSanchez> about raster data: I'll be getting a hard drive full or
> orthophotos, convert them to geotiff, put them into a torrent. Stay tuned.
> <IvanSanchez> jmckenna: is roads with casing asking too much?
> <acuster> seems an sld test would be a resonable thing to add, and one team
> doesn't play in that test
> <jmckenna> IvanSanchez: example styling from last year (see map image
> below): http://wiki.osgeo.org/wiki/Texas_roads_styled
> <sigq> Title: Texas roads styled - OSGeo Wiki (at wiki.osgeo.org)
> <acuster> IvanSanchez, these will already be a mosaic?
> <IvanSanchez> acuster: the raster data will be something *about* 50 files,
> each one covering several square kilometers
> <IvanSanchez> each file is a processed mosaic, if that's what you mean
> <acuster> I'm wondering if they are aligned
> <acuster> or if they overlap
> <IvanSanchez> acuster: I *think* they overlap just a little bit
> <acuster> thanks
> <IvanSanchez> acuster: I will be putting a schematic of how much raster
> data I'll be trying to get
> <IvanSanchez> I think they overlap a little bit in the borders, if just to
> prevent gaps when rendering
> <IvanSanchez> acuster: see
> http://wiki.osgeo.org/wiki/Benchmarking_2010/How_to_get_some_sample_data#Raster_data_issues
> <sigq> Title: Benchmarking 2010/How to get some sample data - OSGeo Wiki
> (at wiki.osgeo.org)
> <IvanSanchez> I've uploaded an image of just how much raster data, and how
> it will be distributed
> <IvanSanchez> basically, one file per square in that diagram
> <IvanSanchez> I will provide size estimates (in GB) for the raster data as
> soon as I start converting ECWs into geotiffs
> <jmckenna> we have reached our meeting time limit of one hour.  I'd like to
> see feedback on data from all teams sent to the email list, so IvanSanchez
> knows what layers to pre-process.  If the teams all want pretty maps then we
> can do pretty maps this year, it depends on what we agree to.  Next week's
> meeting: I will be attending an event (with danmo and msmith_) so I cannot
> attend. i will follow logs...
> <jmckenna> ...and meeting summary.
> <ascollignon> IvanSanchez, as for the best effort ECW will be also use,
> could you provide both ECW and Geotiff? In order to avoid to re-convert
> internaly?
> <acuster> IvanSanchez, thanks for the work
> <IvanSanchez> ascollignon: I can provide both datasets, will probably set
> up a torrent for each
> <acuster> IvanSanchez, are you going to reproject the GeoTiff's also?
> <IvanSanchez> acuster: you're welcome
> <acuster> if not, what are they in?
> <IvanSanchez> acuster: I don't plan to reproject geotiffs - my single-core
> at home should not cope with that
> <acuster> ok
> <IvanSanchez> acuster: download one ECW image (see wiki instructions) and
> see for yourself
> <IvanSanchez> I cannot recall
> * IvanSanchez shuts up
> <acuster> ecw is unfortunately a terrifically encumbered technology
> <acuster> I'll wait for your geotiffs
> <-- marco___ has quit (Remote host closed the connection)
> <msmith_> I won't be on next weeks meeting as jmckenna said, but it would
> be good to decide on server setup so I can get the machines available on the
> net if we are going to use the ones from last year.
> <acuster> msmith_, seems you are the only one with hardware to offer so we
> will probably accept gladly
> <msmith_> I think the only option for the wms server will be as VMs as we
> will want to run each OS on an identical box and this is the only way I can
> think of without swapping drives
> <IvanSanchez> as jmckenna said, I'd like to see feedback from the teams to
> see how pretty do we want our maps
> <acuster> IvanSanchez, that goes along with my desire to know what we will
> be testing
> <acuster> reproject/not, sld/not, ...
> <springmeyer> IvanSanchez: I think for styles its going to take someone
> proposing a specific SLD, like andrea did last year, that can be exactly
> matched
> <springmeyer> otherwise is must be very very simple to keep consistency
> <msmith_> can we use the same SLD on the new data?
> <IvanSanchez> acuster: I do agree. Personally, I'd like to see how WMS
> servers perform on rendering semi-complex simbology: flowing antialiased
> text, cased roads, dashed railways...
> <acuster> the sld on line looks iffy
> <IvanSanchez> msmith_: no, if just because the fields names of the
> shapefiles differ from last year's
> <IvanSanchez> msmith_: they could be tweaked, though
> <acuster> IvanSanchez, the problem with that is that if it's beyond the
> spec, no clients can know how to ask for it
> <msmith_> I meant the symbology. The fields can change
> <IvanSanchez> msmith_: Maybe the SLDs will have to contemplate an extra
> color for the roads or so, but it's doable
> <msmith_> acuster: that can be part of the "best foot" display. Here's the
> base SLD and your "enhanced" non-standard SLD
> <acuster> msmith_, but then what are we doing if one has beautiful maps a
> little bit slower than another who has pretty maps?
> <msmith_> Is anybody going to need SDE support or is just postgres and
> oracle db support sufficient?
> <acuster> hard to compare
> <IvanSanchez> acuster: it's hard to compare *anyway*
> <FrankW> to the best of my knowledge we aren't trying to do any SDE this
> year.
> <msmith_> acuster: thats why we have 2 categories. Base and Best
> <msmith_> FrankWL one less thing for me to configure and install
> <IvanSanchez> yeah. and the best Best wins a beer from me.
> <FrankW> I'm sort of interested in also setting up an ingres instance with
> spatial support though it might only be to compare mapserver
> ingres/oracle/postgis performance.
> <acuster> so each team would have to decide whether to aim for pretty or
> for fast?
> <msmith_> If you can provide the install media, I can install it. OS
> requirements?
> <acuster> oh, for beer we'll definately compete for the pretty then
> <acuster> :-)
> <IvanSanchez> acuster: first, aim for fast (base). When you're done, aim
> for pretty (best)
> <msmith_> FrankW: I'd be curious about that.
> <IvanSanchez> basically, base will be shown on the big screens, while the
> beer is... well, having a beeer in a pub filled with geeks
> <acuster> the 'base' as it is being talked about seems strange
> <acuster> for instance, are we allowed to pyramid the geotiffs?
> <FrankW> it was agreed that pyramid would be built for the base geotiffs.
> <acuster> ah, we have to use a common pyramid?
> <FrankW> not necessarily, no.
> <-- ascollignon (~ascollign at mail.ionicsoft.com) has left #foss4g
> <FrankW> The http://logs.qgis.org/foss4g/%23foss4g.2010-06-09.log isn't up
> to date yet, was there any resolution on where we will get machines to run
> the shootout?
> <sigq> Title: IRC Log - #FOSS4G (at logs.qgis.org)
> <acuster> FrankW, msmith_ is the only one with an offer on the table right
> now, from what I understand
> <acuster> and has upgraded to a T1 line from DSL, if I understood correctly
> <FrankW> Thanks, I did read that part.
> <FrankW> I'm a bit nervous about having to run the main test systems in a
> VM.
> <FrankW> And switching VMs will mean even more contention.
> <FrankW> But perhaps we just have to live with that.
> <msmith_> I'm open to ideas but we only have so many machines available
> <msmith_> And I've got to run. I'll check logs for more.
> <FrankW> yes, of course I understand!
> <-- msmith_ has quit (Quit: Page closed)
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.osgeo.org/pipermail/benchmarking/attachments/20100610/67858a06/attachment-0001.html