[gdal-dev] gdal_translate .vrt issue

David Fawcett david.fawcett at gmail.com
Tue May 20 09:18:37 PDT 2014


As usual Even, spot on!

Thank you for precise diagnosis and the quick solution.  My project is back
on track and the product can be delivered soon.  I am guessing that I could
bump the pool_size up somewhat.  I plan to play around with that value and
maybe get a little more performance (on my old Mac laptop...).

Thanks again,

David.


On Tue, May 20, 2014 at 3:37 AM, Even Rouault
<even.rouault at mines-paris.org>wrote:

> Selon David Fawcett <david.fawcett at gmail.com>:
>
> > I am using gdal_translate and a .vrt to take 115 .jp2 files from the
> USGS,
> > pull them together and then export a lower resolution .jp2.
> >
> > This is with GDAL 1.10 on OSX.
> > gdalinfo --version
> > GDAL 1.10.1, released 2013/08/26
> >
> > When I run gdal_translate, it gets about 30% of the way through the
> process
> > and errors out saying that it can't open one of the .j2w files.  When I
> run
> > gdalinfo on the file, I can open it, and if I run gdal_translate on just
> > that file, I can successfully export a reduced resolution copy.
> >
> > The output of gdalinfo is here:
> > https://gist.github.com/fawcett/beead31741e24394b38e
> >
> > I can also load the .vrt image into QGIS 2.2 and it loads (slowly of
> > course).
>
> Even when you zoom out at 10% like your gdal_translate command ?
>
> >
> > gdal_translate img_index.vrt mpls34.jp2 -outsize 10% 10% -of JP2OpenJPEG
> > -co QUALITY=25 -epo
> > Input file size is 45000, 65000
> > 0...10...20.ERROR 4:
> >
>
> CSLLoad2("/Users/david/projects/mpls_watersheds/data/15TVK830815_201203_0x3000m_CL_1/15TVK830815_201203_0x3000m_CL_1.j2w")
> > failed: unable to open file.
> > ERROR 4:
> >
>
> `/Users/david/projects/mpls_watersheds/data/15TVK830830_201203_0x3000m_CL_1/15TVK830830_201203_0x3000m_CL_1.jp2'
> > not recognised as a supported file format.
> >
> > gdalinfo 15TVK830815_201203_0x3000m_CL_1.jp2
> > Driver: JP2OpenJPEG/JPEG-2000 driver based on OpenJPEG library
> > Files: 15TVK830815_201203_0x3000m_CL_1.jp2
> >        15TVK830815_201203_0x3000m_CL_1.j2w
> > Size is 5000, 5000
> > Coordinate System is:
> > PROJCS["NAD83 / UTM zone 15N",
> >     GEOGCS["NAD83",
> >         DATUM["North_American_Datum_1983",
> >             SPHEROID["GRS 1980",6378137,298.2572221010002,
> >                 AUTHORITY["EPSG","7019"]],
> >             AUTHORITY["EPSG","6269"]],
> >         PRIMEM["Greenwich",0],
> >         UNIT["degree",0.0174532925199433],
> >         AUTHORITY["EPSG","4269"]],
> >     PROJECTION["Transverse_Mercator"],
> >     PARAMETER["latitude_of_origin",0],
> >     PARAMETER["central_meridian",-93],
> >     PARAMETER["scale_factor",0.9996],
> >     PARAMETER["false_easting",500000],
> >     PARAMETER["false_northing",0],
> >     UNIT["metre",1,
> >         AUTHORITY["EPSG","9001"]],
> >     AUTHORITY["EPSG","26915"]]
> > Origin = (483000.000000000000000,4983000.000000000000000)
> > Pixel Size = (0.300000000000012,-0.300000000000186)
> > Image Structure Metadata:
> >   INTERLEAVE=PIXEL
> > Corner Coordinates:
> > Upper Left  (  483000.000, 4983000.000) ( 93d12'56.50"W, 45d 0' 0.87"N)
> > Lower Left  (  483000.000, 4981500.000) ( 93d12'56.32"W, 44d59'12.26"N)
> > Upper Right (  484500.000, 4983000.000) ( 93d11'47.99"W, 45d 0' 1.00"N)
> > Lower Right (  484500.000, 4981500.000) ( 93d11'47.82"W, 44d59'12.39"N)
> > Center      (  483750.000, 4982250.000) ( 93d12'22.16"W, 44d59'36.63"N)
> > Band 1 Block=1024x1024 Type=Byte, ColorInterp=Red
> >   Overviews: 2500x2500, 1250x1250, 625x625, 312x312, 156x156
> >   Overviews: arbitrary
> > Band 2 Block=1024x1024 Type=Byte, ColorInterp=Green
> >   Overviews: 2500x2500, 1250x1250, 625x625, 312x312, 156x156
> >   Overviews: arbitrary
> > Band 3 Block=1024x1024 Type=Byte, ColorInterp=Blue
> >   Overviews: 2500x2500, 1250x1250, 625x625, 312x312, 156x156
> >   Overviews: arbitrary
> >
> > The .vrt file contents are here:
> > https://gist.github.com/fawcett/c8cd5d271b0a89e67779
> >
> > Does anyone have a suggestion on how to troubleshoot this?  I am
> wondering
> > if I am just running out of memory.
>
> I guess you might rather run out of file descriptors. What is the default
> limit
> on Mac : 1024 as on Linux ?
> For each opened JP2 by the OpenJPEG driver, it maintains a file descriptor
> per
> overview level (and for the main level). So in your above case, 6 file
> descriptors.
> The VRT mechanism shouldn't open simultaneously more than
> GDAL_MAX_DATASET_POOL_SIZE=100 datasets
> (
> http://trac.osgeo.org/gdal/browser/trunk/gdal/gcore/gdalproxypool.cpp#L313
> ).
> 100 * 6 = 600... So if you have JP2 images bigger than the one you
> sampled, you
> might have more overview levels. Or possibly the limit on Mac is lower than
> 1024.
> You might want to try setting the GDAL_MAX_DATASET_POOL_SIZE configuration
> option/environmenet variable to a lower value, let's say 10, and see if it
> works
> better.
>
> Even
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20140520/a1627c15/attachment.html>


More information about the gdal-dev mailing list