<html><body><div style="color:#000; background-color:#fff; font-family:HelveticaNeue, Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif;font-size:12pt"><div><span style="font-family: Verdana, Arial, 'Bitstream Vera Sans', Helvetica, sans-serif; font-size: 13px; background-color: transparent;">Hi, I made tests with 'test_ogrsf.exe' and a dbase file.</span><br></div><div style="color: rgb(0, 0, 0); font-size: 13px; font-family: Verdana, Arial, 'Bitstream Vera Sans', Helvetica, sans-serif; background-color: transparent; font-style: normal;"><br></div><div style="color: rgb(0, 0, 0); font-size: 13px; font-family: Verdana, Arial, 'Bitstream Vera Sans', Helvetica, sans-serif; background-color: transparent; font-style: normal;"><span><a href="https://github.com/OSGeo/gdal/pull/24">https://github.com/OSGeo/gdal/pull/24</a></span></div><div style="color: rgb(0, 0, 0); font-size: 13px; font-family: Verdana, Arial, 'Bitstream Vera Sans', Helvetica,
sans-serif; background-color: transparent; font-style: normal;"><br></div><div style="color: rgb(0, 0, 0); font-size: 13px; font-family: Verdana, Arial, 'Bitstream Vera Sans', Helvetica, sans-serif; background-color: transparent; font-style: normal;">Regards</div><div style="color: rgb(0, 0, 0); font-size: 13px; font-family: Verdana, Arial, 'Bitstream Vera Sans', Helvetica, sans-serif; background-color: transparent; font-style: normal;"><br></div><div><br></div><blockquote style="border-left: 2px solid rgb(16, 16, 255); margin-left: 5px; margin-top: 5px; padding-left: 5px;"> <div style="font-family: HelveticaNeue, 'Helvetica Neue', Helvetica, Arial, 'Lucida Grande', sans-serif; font-size: 12pt;"> <div style="font-family: HelveticaNeue, 'Helvetica Neue', Helvetica, Arial, 'Lucida Grande', sans-serif; font-size: 12pt;"> <div dir="ltr"> <hr size="1"> <font size="2" face="Arial"> <b><span style="font-weight:bold;">De:</span></b> Even Rouault
<even.rouault@mines-paris.org><br> <b><span style="font-weight: bold;">Para:</span></b> A Huarte <ahuarte47@yahoo.es> <br><b><span style="font-weight: bold;">CC:</span></b> "gdal-dev@lists.osgeo.org" <gdal-dev@lists.osgeo.org> <br> <b><span style="font-weight: bold;">Enviado:</span></b> Martes 22 de octubre de 2013 14:17<br> <b><span style="font-weight: bold;">Asunto:</span></b> Re: [gdal-dev] Support of filemapping for fast rendering in QGIS...<br> </font> </div> <div class="y_msg_container"><br>Selon A Huarte <<a ymailto="mailto:ahuarte47@yahoo.es" href="mailto:ahuarte47@yahoo.es">ahuarte47@yahoo.es</a>>:<br><br>> Hi, I want improve the rendering of vector features in QGIS. <br>><br>> I have opened a new branch of code in the repository of<br>><br>QGIS(<a href="https://github.com/qgis/QGIS/pull/927%C2%A0//%C2%A0https://github.com/ahuarte47/QGIS/tree/Issue_8725-OGR"
target="_blank">https://github.com/qgis/QGIS/pull/927 // https://github.com/ahuarte47/QGIS/tree/Issue_8725-OGR</a>) and<br>> the drawing tests give me ~3x faster that original code :-)<br><br>On which OS ?<br>Is it the performance gain of QGIS + OGR changes combined ?<br>What is the speed-up in OGR side only ? You could for example use the test_ogrsf<br>utility as a potential benchmark. You have to build explicitely with "make<br>test_ogrsf" / "nmake /f makefile.vc test_ogrsf.exe", in the apps directory of<br>GDAL.<br><br>><br>> I am testing with huge shapefiles (>500.000 records) and I have detect that<br>> the OGR-shapefile provider would be optimized using filemapping (boost<br>> <<a href="http://www.boost.org/" target="_blank">http://www.boost.org/</a>>, or others...)<br>><br>> I wonder if there is any support for FileMapping in gdal or planned.<br><br>There is none currently. This is something that
has been talked about by a few<br>people, but never implemented yet. I actually had a local experiment on Linux<br>but I couldn't observe any real performance gain by using mmap() rather than<br>fread(). Things are maybe different on the Windows side.<br><br>> My<br>> intention would be to redefine the access to the files in this provider with<br>> a new function with "SAHooks" using FileMapping. I provide, as explanation,<br>> an unfinished implementation using boost with which greatly speeds up the<br>> read of files for critical, massive and repetitive processes such as the<br>> painting in a map.<br><br>A more general approach that could benefit to other drivers than shapefile<br>(potentially most GDAL and OGR drivers actually) would be to implement a<br>/vsimemmapped/ virtual file system driver. You have many examples of such<br>drivers in the port subdirectory. And like it is done currently with /vsicache/,<br>the default virtual
file system in cpl_vsil_unix_stdio_64.cpp or<br>cpl_vsil_win32.cpp could be instructed to use /vsicache/ if an env. variable is<br>toggled.<br><br>><br>><br>> On the other hand, this provider may also accelerate with some small changes<br>> that are pending of review in the ticket<br>> (<a href="http://trac.osgeo.org/gdal/ticket/5272" target="_blank">http://trac.osgeo.org/gdal/ticket/5272</a>) that I hope will be evaluated.<br><br>I had skimmed through that one too :<br><br>* Did you evaluate the precision loss of using OGRFastAtof() rather than<br>CPLAtof() ?<br><br>* Are you sure that the performance improvement of replacing memcpy() by *dest =<br>*src is significant ? A decent compiler (GCC ;-)) should optimize that on most<br>platforms anyway. The thing is that there are still people using odd platforms<br>where a non-aligned pointer assingment will cause a SIGBUS, hence the need for<br>memcpy(). If the performance gain is significant
enough we could perhaps have a<br>MEMCPY_DOUBLE macro that on X86 would evaluate to *dest = *ptr and on other<br>platforms would default to memcpy().<br><br>Best regards,<br><br>Even<br><br><br><br></div> </div> </div> </blockquote><div></div> </div></body></html>