[gdal-dev] A couple of new issues with gdal
Stephen Woodbridge
woodbri at swoodbridge.com
Fri Apr 14 12:54:50 PDT 2017
Even,
http://imaptools.com:8080/dl/sew-tmp.tbz
See the README.txt file for an explanation. There is a run.log that
shows what gdal commands were run and various intermediate tif/vrt
files. I chopped out a 900x900 image from a full DOQQ, which is also
included.
For some context, I building a workflow that will be able to fetch
DOQQs, prep them, segmentize them, and then either train or search for
objects using scikit machine learning. This particular part of the
workflow is in the prep stage. A lot of the workflow is being built
around the ability to manage large datasets of imagery. For example
California has something like 10-11,000 DOQQs.
Anyway, let me know if you need more info.
Thanks,
-Steve
On 4/14/2017 2:06 PM, Even Rouault wrote:
> On vendredi 14 avril 2017 13:39:12 CEST Stephen Woodbridge wrote:
>
> > Hi Even,
>
> >
>
> > GDAL 2.1.0, released 2016/04/25
>
> >
>
> > I'm pretty happy with the new workflow so far, but have te following
>
> > minor issues.
>
> >
>
> > The color interp is still problematic with gdalinfo, even though the
>
> > *.aux.xml file looks correct, except it is missing the Alpha band. I
>
> > generated the final tif using a VRT that defined the the color interp.
>
> >
>
> > The second issue is that I'm getting a lot of:
>
> > Warning 1: JPEGLib:Premature end of JPEG file
>
> > messages when I run gdaladdo
>
> Can you provide a fully reproducable (ie all input files and scripts)
> procedure so that others can try easily ? Ideally with smallish files, etc
>
> >
>
> > When I try to display it with imagemagick it complains that it is
>
> > PhotometricInterpretation=4, (ie: a Transparency Mask) which seems to
>
> > make sense since I asked for a mask.
>
> >
>
> > -Steve
>
> >
>
> > I've reduced my processing to the following steps:
>
> >
>
> > 1. create a sobel image from the source tif with the same georeferencing
>
> > 2. gdalwarp it to EPSG:4326 in tempfile2
>
> >
>
> > 3. gdalwarp source tif to EPSG:4326 in tempfile1
>
> > gdalwarp -t_srs EPSG:4326 -dstalpha -co TILED=YES \
>
> > src/33118/m_3311814_nw_11_1_20140513.tif \
>
> > tmp/116-1-m_3311814_nw_11_1_20140513.tif
>
> >
>
> > 4. generate a vrt with
>
> > # a band = [filename, src_band_no, color_interp]
>
> > bands = [ [tempfile1, 1, 'Red'],
>
> > [tempfile1, 2, 'Green'],
>
> > [tempfile1, 3, 'Blue'],
>
> > [tempfile1, 4, 'Gray'], # IR band
>
> > [tempfile2, 1, 'Gray'], # Sobel band to add
>
> > [tempfile1, 5, 'Alpha'] ] # Alpha band
>
> >
>
> > and each VRTRasterBand looks like:
>
> > '''
>
> > <VRTRasterBand dataType="{0}" band="{1:d}">
>
> > <ColorInterp>{2}</ColorInterp>
>
> > <SimpleSource>
>
> > <SourceFilename relativeToVRT="1">{3}</SourceFilename>
>
> > <SourceBand>{4:d}</SourceBand>
>
> > <SrcRect xOff="0" yOff="0" xSize="{5:d}" ySize="{6:d}"/>
>
> > <DstRect xOff="0" yOff="0" xSize="{5:d}" ySize="{6:d}"/>
>
> > </SimpleSource>
>
> > </VRTRasterBand>'''
>
> >
>
> > 5. translate vrt to tiff and compress it
>
> > gdal_translate -co TILED=YES -co JPEG_QUALITY=90 -co COMPRESS=JPEG -co
>
> > INTERLEAVE=BAND -mask 6 --config GDAL_TIFF_INTERNAL_MASK YES \
>
> > tmp/116-2-m_3311814_nw_11_1_20140513.vrt \
>
> > dest/m_3311814_nw_11_1_20140513.tif
>
> >
>
> > $ gdalinfo dest/m_3311814_nw_11_1_20140513.tif
>
> > Driver: GTiff/GeoTIFF
>
> > Files: dest/m_3311805_ne_11_1_20140513.tif
>
> > dest/m_3311805_ne_11_1_20140513.tif.aux.xml
>
> > Size is 7232, 7056
>
> > Coordinate System is:
>
> > GEOGCS["WGS 84",
>
> > DATUM["WGS_1984",
>
> > SPHEROID["WGS 84",6378137,298.257223563,
>
> > AUTHORITY["EPSG","7030"]],
>
> > AUTHORITY["EPSG","6326"]],
>
> > PRIMEM["Greenwich",0],
>
> > UNIT["degree",0.0174532925199433],
>
> > AUTHORITY["EPSG","4326"]]
>
> > Origin = (-118.441851318576212,34.003461706049677)
>
> > Pixel Size = (0.000009839810447,-0.000009839810447)
>
> > Metadata:
>
> > AREA_OR_POINT=Area
>
> > Image Structure Metadata:
>
> > COMPRESSION=JPEG
>
> > INTERLEAVE=BAND
>
> > Corner Coordinates:
>
> > Upper Left (-118.4418513, 34.0034617) (118d26'30.66"W, 34d 0'12.46"N)
>
> > Lower Left (-118.4418513, 33.9340320) (118d26'30.66"W, 33d56' 2.52"N)
>
> > Upper Right (-118.3706898, 34.0034617) (118d22'14.48"W, 34d 0'12.46"N)
>
> > Lower Right (-118.3706898, 33.9340320) (118d22'14.48"W, 33d56' 2.52"N)
>
> > Center (-118.4062706, 33.9687469) (118d24'22.57"W, 33d58' 7.49"N)
>
> > Band 1 Block=256x256 Type=Byte, ColorInterp=Gray
>
> > Overviews: 3616x3528, 1808x1764, 904x882, 452x441, 226x221, 113x111,
>
> > 57x56
>
> > Mask Flags: PER_DATASET
>
> > Overviews of mask band: 3616x3528, 1808x1764, 904x882, 452x441,
>
> > 226x221, 113x111, 57x56
>
> > Band 2 Block=256x256 Type=Byte, ColorInterp=Undefined
>
> > Overviews: 3616x3528, 1808x1764, 904x882, 452x441, 226x221, 113x111,
>
> > 57x56
>
> > Mask Flags: PER_DATASET
>
> > Overviews of mask band: 3616x3528, 1808x1764, 904x882, 452x441,
>
> > 226x221, 113x111, 57x56
>
> > Band 3 Block=256x256 Type=Byte, ColorInterp=Undefined
>
> > Overviews: 3616x3528, 1808x1764, 904x882, 452x441, 226x221, 113x111,
>
> > 57x56
>
> > Mask Flags: PER_DATASET
>
> > Overviews of mask band: 3616x3528, 1808x1764, 904x882, 452x441,
>
> > 226x221, 113x111, 57x56
>
> > Band 4 Block=256x256 Type=Byte, ColorInterp=Undefined
>
> > Overviews: 3616x3528, 1808x1764, 904x882, 452x441, 226x221, 113x111,
>
> > 57x56
>
> > Mask Flags: PER_DATASET
>
> > Overviews of mask band: 3616x3528, 1808x1764, 904x882, 452x441,
>
> > 226x221, 113x111, 57x56
>
> > Band 5 Block=256x256 Type=Byte, ColorInterp=Undefined
>
> > Overviews: 3616x3528, 1808x1764, 904x882, 452x441, 226x221, 113x111,
>
> > 57x56
>
> > Mask Flags: PER_DATASET
>
> > Overviews of mask band: 3616x3528, 1808x1764, 904x882, 452x441,
>
> > 226x221, 113x111, 57x56
>
> > Band 6 Block=256x256 Type=Byte, ColorInterp=Alpha
>
> > Overviews: 3616x3528, 1808x1764, 904x882, 452x441, 226x221, 113x111,
>
> > 57x56
>
> > Mask Flags: PER_DATASET
>
> > Overviews of mask band: 3616x3528, 1808x1764, 904x882, 452x441,
>
> > 226x221, 113x111, 57x56
>
> >
>
> >
>
> > dest/m_3311805_ne_11_1_20140513.tif.aux.xml
>
> > <PAMDataset>
>
> > <PAMRasterBand band="1">
>
> > <ColorInterp>Red</ColorInterp>
>
> > </PAMRasterBand>
>
> > <PAMRasterBand band="2">
>
> > <ColorInterp>Green</ColorInterp>
>
> > </PAMRasterBand>
>
> > <PAMRasterBand band="3">
>
> > <ColorInterp>Blue</ColorInterp>
>
> > </PAMRasterBand>
>
> > <PAMRasterBand band="4">
>
> > <ColorInterp>Gray</ColorInterp>
>
> > </PAMRasterBand>
>
> > <PAMRasterBand band="5">
>
> > <ColorInterp>Gray</ColorInterp>
>
> > </PAMRasterBand>
>
> > </PAMDataset>
>
> >
>
> >
>
> > 6. add overviews to tiff
>
> > gdaladdo -clean -r average dest/33118/m_3311814_nw_11_1_20140513.tif
>
> > 2 4 8 16 32 64 128
>
> >
>
> > I get lots of Warnings:
>
> > Warning 1: JPEGLib:Premature end of JPEG file
>
> >
>
> > ---
>
> > This email has been checked for viruses by Avast antivirus software.
>
> > https://www.avast.com/antivirus
>
> >
>
> > _______________________________________________
>
> > gdal-dev mailing list
>
> > gdal-dev at lists.osgeo.org
>
> > https://lists.osgeo.org/mailman/listinfo/gdal-dev
>
> --
>
> Spatialys - Geospatial professional services
>
> http://www.spatialys.com
>
>
>
> _______________________________________________
> gdal-dev mailing list
> gdal-dev at lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/gdal-dev
>
---
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus
More information about the gdal-dev
mailing list