[Gdal-dev] gtiff file size compared to original tiff +wrld file

Thu Apr 23 16:35:45 EDT 2009

Christian and others

just to post an update that others might find useful.

Using the shell command line i used

For %I in C:\Data\250k\*.tif do gdal_translate –of gtiff –co “COMPRESS=LZW”
%I C:\Data\250k\geotiffs\%~nI.tif

this ran through the entire folder translating my tiff_world files in LZW
compression to geotiffs with LZW compression. it worked v well and got the
same output size as the inputs (as you woudl expect)

then because of the file sizes i used gdalwarp rather than gdal_merge.py (as
i seen other posts where people had issues with large rasters)

i originally ran just using

Gdalwarp C:\Data\250k\geotiffs\*.tif  C:\Data\250k\geotiffs\merge.tif

and it took 5hrs and 40mins to process and gave me a merged ratser of 1.5gb
- i noticed my error, i forgot to add the COMPRESS option

so ran it again using the compression to LZW, this took 3hrs for the 56
files and resulted in a 675mb file. this i felt was still much larger than
the original files.

i used gdalinfo to check it was compressed, and the info said it was

so for the fun of it i ran a quick gdal_translate C:\...\merge.tif
C:\......\mosaic.tif -co "COMPRESS=YES"

and the results was a 147mb mosaic.

so i am not sure what the code is behind the gdalwarp compared to the
gdal_translate but would have thought they should compress the same.

if anyone knows why i get this difference would be great to know as it would
remove an extra step from the batch file.

Simone - i posted here rather than over at Geoserver because it was the gdal
that i couldnt seem to get working rather than the method of loading to
geoserver.

Simone Giannecchini-2 wrote:
> 
> Ciao Tim,
> you should probably post these questions to the geoserver users mailing
> list.
> 
> Simone.
> -------------------------------------------------------
> Ing. Simone Giannecchini
> GeoSolutions S.A.S.
> Owner - Software Engineer
> Via Carignoni 51
> 55041  Camaiore (LU)
> Italy
> 
> phone: +39 0584983027
> fax:      +39 0584983027
> mob:    +39 333 8128928
> 
> 
> http://www.geo-solutions.it
> http://simboss.blogspot.com/
> http://www.linkedin.com/in/simonegiannecchini
> 
> -------------------------------------------------------
> 
> 
> 
> On Wed, Apr 22, 2009 at 10:49 AM, TJMartin <tjmgis at gmail.com> wrote:
>>
>> Hi christian
>>
>> many thanks for your help, i will run this later on today/overnight :)
>>
>> I saw that Geoserver tutorail using the database to store the files but
>> think i will stick with using just files.
>>
>> will postback later on if and when i get it all running
>>
>> thanks again
>>
>> Tim
>>
>>
>>
>> Christian Müller wrote:
>>>
>>> Hi, I did this process last year many times and here is my experience
>>>
>>> 1) I assume you omitted the create options, your original tiff files are
>>> compressed, the new ones not.
>>> Look here http://www.gdal.org/frmt_gtiff.html, on the commandline it is
>>> -co
>>> "COMPRESS=JPEG" as an example.
>>>
>>> 2) Take a look at your tiles with gdalinfo, do the have a color table or
>>> more bands ?. If your tiles have
>>> more bands, (e. g. one for read, one for green and one for blue), use
>>> gdal_merge.py with -pct to produce a big picture
>>> having only one band and a color table. Create your tiles from the big
>>> picture. Geoserver will say thank you in terms
>>> of memory usage and cpu performance.
>>>
>>> 3) Decide, wich Geoserver module to use. There are 2 of them. There is a
>>> file based approach and a jdbc based
>>> approach (Module imagemosic-jdbc). If you want to store the whole stuff
>>> in
>>> a
>>> database, use gdal_retile.py to procude your
>>> tiles and pyramids and the imagemosic-jdbc module to import all the
>>> tiles.
>>> Both, gdal_retily.py and the imagemosaic-jdbc
>>> module were developed by me, if you have questions, ask.
>>>
>>> The documentation is here
>>> http://docs.codehaus.org/display/GEOTDOC/Image+Mosaicing+Pyramidal+JDBC+Plug
>>> in
>>> A tutorial for geoserver is here
>>> http://geoserver.org/display/GEOSDOC/Using+the+ImageMosaic+JDBC+Plugin
>>>
>>> christian
>>>
>>>
>>> TJMartin writes:
>>>
>>>>
>>>> Hi All
>>>>
>>>> I am just getting started with gdal, and i have done a search but
>>>> couldnt
>>>> find an answer.
>>>>
>>>> I currently have 10,000 tiff files with their world files. My real aim
>>>> is
>>>> to
>>>> tile them up and make them easier to use in Geoserver or create an
>>>> image
>>>> pyramid.
>>>>
>>>> So i used gdal_translate to Geotiff, then was going to use gdalwarp to
>>>> mosaic, then tile them.
>>>>
>>>> However the file size for geotiffs are between 4 and 15 times the size
>>>>
>>>> 4mb to 60mb
>>>>
>>>> One set were originally 128mb and now 858mb.
>>>>
>>>> So if i was to mosiac these and then tile them it will still be a huge
>>>> data/size volume to server via geoserver.
>>>>
>>>>
>>>> So my questions
>>>>
>>>> 1) Can i reduce the quality when translating to geotiff, and how much
>>>> will
>>>> this affect the image quality when i mosaic and tile
>>>>
>>>> 2) Should i mosaic the tiff + world files instead and then tile them
>>>> with
>>>> world files instead of using geotiffs
>>>>
>>>> 3) is there  a better way of doing this?
>>>>
>>>> would really appreciate anyone's help
>>>>
>>>> all the best
>>>>
>>>> Tim
>>>>
>>>> --
>>>> View this message in context:
>>>> http://n2.nabble.com/gtiff-file-size-compared-to-original-tiff-%2Bwrld-file-tp2672921p2672921.html
>>>> Sent from the GDAL - Dev mailing list archive at Nabble.com.
>>>>
>>>> _______________________________________________
>>>> gdal-dev mailing list
>>>> gdal-dev at lists.osgeo.org
>>>> http://lists.osgeo.org/mailman/listinfo/gdal-dev
>>>
>>>
>>> _______________________________________________
>>> gdal-dev mailing list
>>> gdal-dev at lists.osgeo.org
>>> http://lists.osgeo.org/mailman/listinfo/gdal-dev
>>>
>>>
>>
>> --
>> View this message in context:
>> http://n2.nabble.com/gtiff-file-size-compared-to-original-tiff-%2Bwrld-file-tp2672921p2674970.html
>> Sent from the GDAL - Dev mailing list archive at Nabble.com.
>>
>> _______________________________________________
>> gdal-dev mailing list
>> gdal-dev at lists.osgeo.org
>> http://lists.osgeo.org/mailman/listinfo/gdal-dev
>>
> _______________________________________________
> gdal-dev mailing list
> gdal-dev at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/gdal-dev
> 
> 

-- 
View this message in context: http://n2.nabble.com/gtiff-file-size-compared-to-original-tiff-%2Bwrld-file-tp2672921p2686259.html
Sent from the GDAL - Dev mailing list archive at Nabble.com.