<br><br><div class="gmail_quote">On Tue, May 28, 2013 at 6:57 PM, Etienne Tourigny <span dir="ltr"><<a href="mailto:etourigny.dev@gmail.com" target="_blank">etourigny.dev@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br><br><div class="gmail_quote"><div class="im">On Sun, May 26, 2013 at 11:46 AM, Even Rouault <span dir="ltr"><<a href="mailto:even.rouault@mines-paris.org" target="_blank">even.rouault@mines-paris.org</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Hi,<br>
<br>
I've just commited in trunk a new file port/vsipreload.cpp.<br>
<br>
This file is the source code for a standalone shared library that can be<br>
LD_PRELOAD'ed as an overload of libc to enable VSI Virtual FILE API to be used<br>
with binaries using regular libc I/O API.<br>
<br>
WARNING: Linux glibc ONLY. Might work with some adaptations (mainly around<br>
64bit symbols) on other Unix systems<br>
<br>
Compile:<br>
g++ -Wall -fPIC port/vsipreload.cpp -shared -o vsipreload.so -Iport \<br>
-L. -L.libs -lgdal<br>
<br>
Examples :<br>
<br>
LD_PRELOAD=vsipreload.so gdalinfo \<br>
/vsicurl/<a href="http://download.osgeo.org/gdal/data/ecw/spif83.ecw" target="_blank">http://download.osgeo.org/gdal/data/ecw/spif83.ecw</a></blockquote><div><br></div></div><div>Even,</div><div><br></div><div>I have built vsipreload with a fresh build of svn trunk, but I get the following error:</div>
<div><br></div><div>$ g++ -Wall -fPIC port/vsipreload.cpp -shared -o vsipreload.so -Iport -L. -L.libs -lgdal</div><div>$ LD_PRELOAD=vsipreload.so gdalinfo /vsicurl/<a href="http://download.osgeo.org/gdal/data/ecw/spif83.ecw" target="_blank">http://download.osgeo.org/gdal/data/ecw/spif83.ecw</a></div>
<div>ERROR: ld.so: object 'vsipreload.so' from LD_PRELOAD cannot be preloaded: ignored.</div><div>ERROR 4: `/vsicurl/<a href="http://download.osgeo.org/gdal/data/ecw/spif83.ecw" target="_blank">http://download.osgeo.org/gdal/data/ecw/spif83.ecw</a>' not recognised as a supported file format.</div>
<div><br></div><div>gdalinfo failed - unable to open '/vsicurl/<a href="http://download.osgeo.org/gdal/data/ecw/spif83.ecw" target="_blank">http://download.osgeo.org/gdal/data/ecw/spif83.ecw</a>'.</div><div><br></div>
<div>Is there a way to find out why it cannot be preloaded? Perhaps the compilation options should be tweaked?</div></div></blockquote><div><br></div><div>I have managed to get it working buy using LD_PRELOAD=./vsipreload.so </div>
<div><br></div><div>e.g.</div><div><div>LD_PRELOAD=./vsipreload.so gdalinfo /vsicurl/<a href="http://download.osgeo.org/gdal/data/netcdf/utm-big-chunks.nc">http://download.osgeo.org/gdal/data/netcdf/utm-big-chunks.nc</a></div>
</div><div><br></div><div>It seems that setting LD_PRELOAD outside of the command works also, so this mechanism would be used transparently.</div><div>But does it have a significant overhead and/or chances it might do damage if always set?</div>
<div><br></div><div>e.g.</div><div><div>export LD_PRELOAD=./vsipreload.so </div><div>gdalinfo /vsicurl/<a href="http://download.osgeo.org/gdal/data/netcdf/utm-big-chunks.nc">http://download.osgeo.org/gdal/data/netcdf/utm-big-chunks.nc</a></div>
</div><div><br></div><div><br></div><div>Is there any way this could be integrated into gdal to transparently support vsifile for all drivers, without setting LD_PRELOAD?</div><div>In this way this feature would be available to all gdal commands or even other apps that use gdal. </div>
<div><br></div><div>For example, /vsizip/ is used in QGIS, but it does not work with certain files (e.g. netcdf ) inside archives. But if I use LD_PRELOAD it works for netcdf files! Neat stuff!</div><div>e.g. LD_PRELOAD=./vsipreload.so qgis</div>
<div><br></div><div>great work</div><div>Etienne</div><div><br></div><div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="gmail_quote">
<div><br></div><div>Thanks,</div><div>Etienne</div><div><br></div><div>My system is Linux Mint 13 ubuntu (quantal), gcc version 4.7.2, 64bits, 3.5.0-30-generic kernel</div><div class="im"><div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>
<br>
LD_PRELOAD=vsipreload.so gdalinfo \<br>
'HDF4_EOS:EOS_GRID:"/vsicurl/<a href="http://download.osgeo.org/gdal/data/hdf4/MOD09Q1G_EVI.A2006233.h07v03.005.2008338190308.hdf" target="_blank">http://download.osgeo.org/gdal/data/hdf4/MOD09Q1G_EVI.A2006233.h07v03.005.2008338190308.hdf</a>":MODIS_NACP_EVI:MODIS_EVI'<br>
<br>
LD_PRELOAD=vsipreload.so ogrinfo \<br>
/vsicurl/<a href="http://svn.osgeo.org/gdal/trunk/autotest/ogr/data/testavc" target="_blank">http://svn.osgeo.org/gdal/trunk/autotest/ogr/data/testavc</a> -ro<br>
<br>
LD_PRELOAD=vsipreload.so ogrinfo -ro \<br>
<br>
/vsizip//vsicurl/<a href="http://download.osgeo.org/gdal/1.10.0/gdalautotest-1.10.0.zip/gdalautotest-1.10.0/ogr/data/testavc" target="_blank">http://download.osgeo.org/gdal/1.10.0/gdalautotest-1.10.0.zip/gdalautotest-1.10.0/ogr/data/testavc</a><br>
<br>
It can work even with non GDAL binaries :<br>
<br>
LD_PRELOAD=vsipreload.so h5dump -d /x \<br>
/vsicurl/<a href="http://download.osgeo.org/gdal/data/netcdf/utm-big-chunks.nc" target="_blank">http://download.osgeo.org/gdal/data/netcdf/utm-big-chunks.nc</a><br>
<br>
LD_PRELOAD=vsipreload.so sqlite3 \<br>
/vsicurl/<a href="http://download.osgeo.org/gdal/data/sqlite3/polygon.db" target="_blank">http://download.osgeo.org/gdal/data/sqlite3/polygon.db</a> \<br>
"select * from polygon limit 10"<br>
<br>
This can work with all VSI Large File API filesystems : /vsizip/ , /vsitar/,<br>
/vsisubfile/ , etc...<br>
<br>
This is still a bit experimental in the sense that only the most common glibc<br>
I/O API have been overloaded. If exotic ones are used with /vsi files, crashes<br>
are likely.<br>
<span><font color="#888888"><br>
--<br>
Geospatial professional services<br>
<a href="http://even.rouault.free.fr/services.html" target="_blank">http://even.rouault.free.fr/services.html</a><br>
_______________________________________________<br>
gdal-dev mailing list<br>
<a href="mailto:gdal-dev@lists.osgeo.org" target="_blank">gdal-dev@lists.osgeo.org</a><br>
<a href="http://lists.osgeo.org/mailman/listinfo/gdal-dev" target="_blank">http://lists.osgeo.org/mailman/listinfo/gdal-dev</a><br>
</font></span></blockquote></div></div><br>
</blockquote></div><br>