[Qgis-developer] Processing with raster files > 4GB

[Andreas Neumann]

Hi,

I was able to do my color replacements with gdal_calc.py. It works fast 
and efficient.

Here is my syntax:

gdal_calc.py -A image.tif --outfile=image_output.tif 
--calc="(A==249)*255+(A!=249)*A"

Syntax looks similar to the QGIS raster calculator - but as opposed to 
the QGIS raster calculator it actually works!

I will still continue to find out about SAGA. Maybe directly converting 
from TIFF to SAGA before processing files in SAGA.

I believe that the QGIS raster calculator would need some developer love 
- to make it more robust and easier to use ;-)

I am also pretty sure that the Processing integration of GRASS/SAGA does 
not work well with big files - at least on Windows and when TIFF files 
are used.

Thanks to all who helped!
Andreas


Am 2014-11-19 11:01, schrieb Carlos López PSIG:
> Hi Andreas,
> 
> Yes, you can use gdal_calc.py
> Certainly you need to perform several processes, but are very fast!
> 
> Be careful with bands!
> Perhaps you will want to separate them.
> 
> Ciao,
> 
>  [3]
> 
> CARLOS LÓPEZ QUINTANILLA
> 
> www.psig.es [3]
> carlos.lopez at psig.es
> +34 699.680.261
> 
> 2014-11-19 9:26 GMT+01:00 Andreas Neumann <a.neumann at carto.net>:
> 
>> Hi Carlos,
>> 
>> GDAL would be my first choice - because it easy to automate.
>> However, I don't know how can do my task with GDAL.
>> 
>> From my grayscale tiff images I want to replace one pixel value
>> (249) with white (255). All other pixels should stay at the value
>> they have.
>> 
>> I came across https://github.com/chiatt/gdal_reclassify [1] - but
>> this reclassifies all pixels, not only selected ones.
>> 
>> Maybe I could use http://www.gdal.org/gdal_calc.html [2] with a
>> conditional syntax?
>> 
>> Thanks,
>> Andreas
>> 
>> Am 2014-11-19 09:10, schrieb Carlos López PSIG:
>> Hi Andreas,
>> 
>> I also had problems to reclassify BIGTIFF with SAGA and GRASS!
>> Finally I used GDAL!
>> 
>> Best regards,
>> 
>>  [2]
>> 
>> CARLOS LÓPEZ QUINTANILLA
>> 
>> www.psig.es [3] [2]
>> 
>> carlos.lopez at psig.es
>> +34 699.680.261 [4]
>> 
>> 2014-11-19 9:05 GMT+01:00 Andreas Neumann <a.neumann at carto.net>:
>> 
>> Hi,
>> 
>> I tried various tools, e.g.
>> 
>> * SAGA reclassify
>> * GRASS r.mapcalc
>> 
>> Both work fine on smaller data sets, but fail on big data sets.
>> 
>> This is on Win7 64bit, with the OSGeo4W installer. I have 8gb RAM
>> and 40GB disk free on C:, more free space on other disks where the
>> data resides.
>> 
>> I am not totally sure if it is the Bigtiff issue or some other
>> issue. It is just a guess that maybe I am hitting a limit with
>> tiff.
>> It is not fully transparent to me what Processing is doing in the
>> background (what file formats with what options it creates).
>> 
>> It may also be an issue with SAGA or GRASS directly.
>> 
>> Thanks if you know anything,
>> Andreas
>> 
>> Am 2014-11-18 21:15, schrieb Giovanni Manghi:
>> 
>> Hi,
>> 
>> I am having a lot of troubles with large raster files and
>> processing.
>> 
>> The intermediate results can go >4GB. Processing stops with not so
>> helpful error messages.
>> 
>> I wonder if processing is hitting the 4GB limit. Can I force
>> processing
>> to use the BIGTIFF option or a raster format that can handle >4GB
>> files?
>> 
>> Hi Andreas, what tool specifically?
>> 
>> cheers
>> 
>> -- Giovanni --
>> _______________________________________________
>> Qgis-developer mailing list
>> Qgis-developer at lists.osgeo.org
>> http://lists.osgeo.org/mailman/listinfo/qgis-developer [5] [1]
>>  _______________________________________________
>>  Qgis-developer mailing list
>>  Qgis-developer at lists.osgeo.org
>>  http://lists.osgeo.org/mailman/listinfo/qgis-developer [5] [1]
>> 
>> Links:
>> ------
>> [1] http://lists.osgeo.org/mailman/listinfo/qgis-developer [5]
>> [2] http://www.psig.es [3]
> 
> 
> 
> Links:
> ------
> [1] https://github.com/chiatt/gdal_reclassify
> [2] http://www.gdal.org/gdal_calc.html
> [3] http://www.psig.es
> [4] tel:%2B34%20699.680.261
> [5] http://lists.osgeo.org/mailman/listinfo/qgis-developer