[gdal-dev] mosaicking files with partial overlap and binary mask

lars.schylberg at blixtmail.se lars.schylberg at blixtmail.se
Thu Feb 11 11:25:21 PST 2021 

Maybe this could help a little.

I had a problem a while ago when I wanted to filter just the mask and then assemble it again. It took some reading to get it right.
So I took the image apart with gdal_translate -b mask. Then ran gdal_sieve.py where I got a 8 bit mask.
The tricky part was then the assembly. This is how I did the reassemble with two gdalbuildvrt:

### Convert the mask to a real 1 bit mask with the right mask flags
### and deflate compression

echo "gdal_translate - real mask"

gdal_translate 
 -of GTiff 
 -ot Byte 
 -co NBITS=1 
 -co COMPRESS=DEFLATE 
 -mo INTERNAL_MASK_FLAGS_1=2 
 ${TIF_TMP_DIR}/${TIF_FILE_NAME}_mask_separate_clean.tif 
 ${TIF_TMP_DIR}/${TIF_FILE_NAME}_mask.msk

### Combine the image with the mask

gdalbuildvrt -b 1 ${TIF_TMP_DIR}/${TIF_FILE_NAME}_warp_1.vrt ${TIF_TMP_DIR}/${TIF_FILE_NAME}_warp.tif
gdalbuildvrt -b 2 ${TIF_TMP_DIR}/${TIF_FILE_NAME}_warp_2.vrt ${TIF_TMP_DIR}/${TIF_FILE_NAME}_warp.tif
gdalbuildvrt -b 3 ${TIF_TMP_DIR}/${TIF_FILE_NAME}_warp_3.vrt ${TIF_TMP_DIR}/${TIF_FILE_NAME}_warp.tif

echo "gdalbuildvrt"

gdalbuildvrt 
 -separate ${TIF_TMP_DIR}/${TIF_FILE_NAME}_combined.vrt 
 ${TIF_TMP_DIR}/${TIF_FILE_NAME}_warp_1.vrt 
 ${TIF_TMP_DIR}/${TIF_FILE_NAME}_warp_2.vrt 
 ${TIF_TMP_DIR}/${TIF_FILE_NAME}_warp_3.vrt 
 ${TIF_TMP_DIR}/${TIF_FILE_NAME}_mask.msk

echo "final gdal_translate"

gdal_translate 
 -b 1 -b 2 -b 3 -mask 4 
 -co COMPRESS=NONE 
 -co TILED=YES 
 -co BLOCKXSIZE=512 
 -co BLOCKYSIZE=512 
 -co INTERLEAVE=BAND 
 --config GDAL_TIFF_INTERNAL_MASK YES 
 --config GDAL_CACHEMAX 512 
 ${TIF_TMP_DIR}/${TIF_FILE_NAME}_combined.vrt 
 ${DATA_OUTPUT_DIR}/${TIF_FILE_NAME}.tif

Hope this helps you to take the image apart and then put back together again.

/Lars S.
11 februari 2021 kl. 18:37, "Patrick Young" <patrick.mckendree.young at gmail.com (mailto:patrick.mckendree.young at gmail.com?to=%22Patrick%20Young%22%20<patrick.mckendree.young at gmail.com>)> skrev:
 Not sure what the expected behavior is with gdalwarp and the underlying mask preservation. It is kind of a tricky problem to solve in general; should the mask be resampled with the same kernel as used in the warp? Or should it always be NN? If the functionality is missing, this is a good opportunity to get your employer funding gdal development.
A quick hack would be like you mention; you can extract the datamask (gdal_translate -b mask ... i think) and then warp that separately, and recombine. 
P
 On Thu, Feb 11, 2021 at 7:57 AM Daniele Romagnoli <daniele.romagnoli at geo-solutions.it (mailto:daniele.romagnoli at geo-solutions.it)> wrote:  Hi Jukka,
sorry but I can't share the data. Is owned by a customer :( 
Long story short: 
I have several raster tiles (with size around 5000x5000) and a shapefile containing edges defining valid area of each tile. 
I have used gdalrasterize to extract the needed crop for each tile to setup a binary mask. 
Then, I have combined the binary mask with the original tile and rewritten as a new TIFF file with internal masks. 
Then, I'm composing all the TIFF files to a VRT. 
When rewriting that VRT to a mosaic TIFF, everything looks good. I see the binary mask is used to properly compose/overlap/mask the pieces of the mosaic and the output contains the mask too. This is an example of gdalinfo on one of the mosaicked tiff: 
Driver: GTiff/GeoTIFF
Files: out.tif
Size is 45842, 42506
Coordinate System is:
[... OMITTED... ]

Metadata:
AREA_OR_POINT=Area
Image Structure Metadata:
COMPRESSION=YCbCr JPEG
INTERLEAVE=PIXEL
JPEGTABLESMODE=1
JPEG_QUALITY=75
SOURCE_COLOR_SPACE=YCbCr
Corner Coordinates:
[... OMITTED... ]
Band 1 Block=256x256 Type=Byte, ColorInterp=Red
Mask Flags: PER_DATASET
Band 2 Block=256x256 Type=Byte, ColorInterp=Green
Mask Flags: PER_DATASET
Band 3 Block=256x256 Type=Byte, ColorInterp=Blue
Mask Flags: PER_DATASET 
So, as you can see the masks are added to the mosaicked dataset. 
However, when running 
gdalwarp -t_srs EPSG:NEWCODE out.tif warped.tif 
the binary mask is lost from the output file. 
I have also tried using -dstalpha but it complains that "Destination dataset has 3 bands, but at least 4 are needed" which makes sense. 
So I was just wondering if there is anything I should be aware of when warping a masked TIFF. 
I would probably try to transform the binary mask to an alpha band, do the warp and then convert back the alpha to a binary mask.... 
Thanks for any feedback in advance. 
Regards, 
Daniele 
 On Thu, Feb 11, 2021 at 3:27 PM jratike80 <jukka.rahkonen at maanmittauslaitos.fi (mailto:jukka.rahkonen at maanmittauslaitos.fi)> wrote: Hi Daniele,

Test data, please.

-Jukka Rahkonen-
Daniele Romagnoli-3 wrote
> Thanks for the feedback, Patrick.
> I have downloaded the latest build from GisInternals and it worked as
> expected.
> I have another question.
> It looks like warping an input file containing a binary mask isn't
> preserving the binary mask on the output.
> Should I specify any flag for that?
>
> Thanks for the help.
> Regards,
> Daniele
> _______________________________________________
> gdal-dev mailing list

> gdal-dev at .osgeo

> https://lists.osgeo.org/mailman/listinfo/gdal-dev (https://lists.osgeo.org/mailman/listinfo/gdal-dev)
--
Sent from: http://osgeo-org.1560.x6.nabble.com/GDAL-Dev-f3742093.html (http://osgeo-org.1560.x6.nabble.com/GDAL-Dev-f3742093.html)
_______________________________________________
gdal-dev mailing list
gdal-dev at lists.osgeo.org (mailto:gdal-dev at lists.osgeo.org)
https://lists.osgeo.org/mailman/listinfo/gdal-dev (https://lists.osgeo.org/mailman/listinfo/gdal-dev)   
--
 Regards, Daniele Romagnoli == GeoServer Professional Services from the experts! Visit http://goo.gl/it488V (http://goo.gl/it488V) for more information. ==  Ing. Daniele Romagnoli Senior Software Engineer  GeoSolutions S.A.S. Via di Montramito 3/A 55054 Massarosa (LU) Italy phone: +39 0584 962313 fax: +39 0584 1660272  http://www.geo-solutions.it (http://www.geo-solutions.it) http://twitter.com/geosolutions_it (http://twitter.com/geosolutions_it)   -------------------------------------------------------  Con riferimento alla normativa sul trattamento dei dati personali (Reg. UE 2016/679 - Regolamento generale sulla protezione dei dati “GDPR”), si precisa che ogni circostanza inerente alla presente email (il suo contenuto, gli eventuali allegati, etc.) è un dato la cui conoscenza è riservata al/i solo/i destinatario/i indicati dallo scrivente. Se il messaggio Le è giunto per errore, è tenuta/o a cancellarlo, ogni altra operazione è illecita. Le sarei comunque grato se potesse darmene notizia.  This email is intended only for the person or entity to which it is addressed and may contain information that is privileged, confidential or otherwise protected from disclosure. We remind that - as provided by European Regulation 2016/679 “GDPR” - copying, dissemination or use of this e-mail or the information herein by anyone other than the intended recipient is prohibited. If you have received this email by mistake, please notify us immediately by telephone or e-mail.   _______________________________________________
gdal-dev mailing list
gdal-dev at lists.osgeo.org (mailto:gdal-dev at lists.osgeo.org)
https://lists.osgeo.org/mailman/listinfo/gdal-dev (https://lists.osgeo.org/mailman/listinfo/gdal-dev)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20210211/0e763699/attachment.html>

More information about the gdal-dev mailing list