Gdal Bug? was: Re: [gdal-dev] JP2MRSID Issues
woodbri at swoodbridge.com
Thu Nov 13 00:46:39 EST 2008
Frank Warmerdam wrote:
> Stephen Woodbridge wrote:
>> Frank, anyone,
>> Thoughts on this issue?
>> Thoughts on how to better diagnose the issue?
>> I can talk to the client about making an image available to someone if
>> that would help.
>> I'm totally stuck on this one, converting 300+ images to tif first at
>> 2-3 hours per image is not going to hack it.
>>>> gdalwarp: ../compressed/compressed.cpp:1724: bool
>>>> kd_tile::read_tile_part_header(): Assertion `tpart_body_length >= 0'
> As I understand it, the problem you see is that under some access
> arrangements you are getting this kd_tile assertion from the JP2MRSID
> driver. Is that right? I'm not familiar with this error, and it is coming
> from deep inside the MrSID SDK (in fact within the subcomponent that is
> derived from the Kakadu library as we can see from the names).
> My impression is that to make any progress you would need to supply
> a sample that demonstrates the problem. At that point I could try and
> diagnose if I'm doing something boneheaded. If nothing is apparently
> someone would need to bounce the issue (hopefully with a narrow example
> program operating directly against the MrSID SDK) to Lizardtech for further
> No offense to Lizardtech - we have vaguely similar issues with ECW,
> and Kakadu too - but it is just hellish to diagnose some of these problems
> deep in proprietary and/or complicated sub-libraries. I find it hard to
> bring myself to even dig into such issues unless a paying client makes me
> do it, though I'd be willing to make a quick try for you if there was an
> easy way for me to try.
Thank you for your response. I know you are busy and I really appreciate
your help and suggestions.
I understand your reluctance on diving into vendor code issues, I feel
pretty much the same way, but I also feel like I need to offer some
options to the client. I will discuss having them talk to you about
funding this if there are no other options.
> PS. My daily download limit via my satellite ISP is 500MB and I don't want
> to use it all up on a single jp2 file. Please try to reproduce the issue
> with a small file.
I'm still looking for a smaller file that I can reproduce this issue on.
The smallest file I have at 92MB worked fine, so I need to try a few
more. The file below at 342MB exhibits the problem. Looking for
> PPS. I don't understand your point where you seem to suggest that
> to tiff first is slower than other mechanisms. Are you suggesting it takes
> 2-3 hours to convert a 500MB jpeg2000 image to GeoTIFF? Perhaps you need
> to up your GDAL block cache size?
Yes, this is exactly my problem it is taking 2-3 hours to run:
GDAL_SKIP=JPEG2000 gdal_translate my.jp2 -b 1 -b 2 -b 3 my.tif
The above was a replacement to for creating a VRT file which was very
GDAL_SKIP=JPEG2000 gdal_translate -of VRT my.jp2 -b 1 -b 2 -b 3 my.vrt
This is a pretty beefy system with 6GB of memory and fast disks, etc.
How do I up the GDAL block cache size? Please give me a hint or point me
to the right page(s).
I have also tried:
[swoodbridge at booboo ~]$ ls -lh
-rw-rw-r-- 1 swoodbridgeusers 342M Oct 20 16:49
[swoodbridge at booboo ~]$ GDAL_SKIP=JPEG2000 gdal_translate --config
/var/data/raw_data/minnesota/ortho_1-1_1m_j_mn161_2008_1.jp2 -b 1 -b 2
-b 3 my.tif
Input file size is 36010, 49447
and it does not seem to be any faster, but I confess to not having
waited very long 2-5mins max.
I also tried the following which was a little faster at 43 mins:
GDAL_SKIP=JPEG2000 gdalwarp --config GDAL_CACHEMAX 500 -wm 500 -co
"TILED=YES" my.vrt my.tif
Creating output file that is 36010P x 49447L.
Processing input file my.vrt.
0...10...20...30...40...50...60...70...80...90...100 - done.
More information about the gdal-dev