[gdal-dev] gdalwarp Resampling Input Raster Nodata Setting
Chao Li
chaoli0394 at gmail.com
Mon Jun 27 23:31:22 PDT 2022
Hi,
Thank you for your reply. I realized the issues. Thank you.
Li
Rahkonen Jukka <jukka.rahkonen at maanmittauslaitos.fi> 于2022年6月28日周二 15:27写道：
> Hi,
>
>
>
> You are reading the documentation optimistically. The -srsnodata does
> accept a list but the list can only be a one nodata value per band.
>
> “Set nodata masking values for input bands (different values can be
> supplied for each band).” In the GDAL datamodel the nodata in rasterband is
> a single value https://gdal.org/api/gdalrasterband_cpp.html. In addition,
> all image formats do not support having a separate nodata value for each
> band. GeoTIFF for example is using the same one nodata value for all bands
> https://gdal.org/drivers/raster/gtiff.html “GDAL stores band nodata value
> in the non standard TIFFTAG_GDAL_NODATA ASCII tag (code 42113) for files
> created with the default profile GDALGeoTIFF. Note that all bands must use
> the same nodata value.”
>
>
>
> There is a similar question in gis.stackexchange from couple of years ago
> https://gis.stackexchange.com/questions/380092/gdalwarp-assigning-more-than-one-nodata-value
> with an advice about how to update the raster with Python to have a single
> value for nodata.
>
>
>
> -Jukka Rahkonen-
>
>
>
> *Lähettäjä:* Chao Li <chaoli0394 at gmail.com>
> *Lähetetty:* tiistai 28. kesäkuuta 2022 2.58
> *Vastaanottaja:* Rahkonen Jukka <jukka.rahkonen at maanmittauslaitos.fi>
> *Kopio:* gdal-dev at lists.osgeo.org
> *Aihe:* Re: [gdal-dev] gdalwarp Resampling Input Raster Nodata Setting
>
>
>
> Hi,
>
>
>
> I read the gdalwarp synopsis: https://gdal.org/programs/gdalwarp.html
> <https://eur06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgdal.org%2Fprograms%2Fgdalwarp.html&data=05%7C01%7Cjukka.rahkonen%40maanmittauslaitos.fi%7C7fbed8ed633f467be75a08da5898e96a%7Cc4f8a63255804a1c92371d5a571b71fa%7C0%7C0%7C637919711038387709%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=olfoXnii%2FhSi6RHqcF6c%2BCfM78eFEAeRG1hJNK2VSMA%3D&reserved=0>
>
>
>
>
> I find the multi values can be used, but it has to write in -srcnodata
> "32760 32761 32762 32763 32764 32765 32766 32767".
>
>
>
> However, it does not work. Does anyone know how to figure it out?
>
>
>
> Thanks
>
>
>
> Rahkonen Jukka <jukka.rahkonen at maanmittauslaitos.fi> 于2022年6月27日周一 20:11
> 写道：
>
> Hi,
>
>
>
> Your idea is OK in theory but GeoTIFF supports only one nodata value. This
> is not relevant with your single band data, but it may be good to know that
> in GeoTIFF the nodata value is also the same for all bands. Therefore with
> GeoTIFF it is not possible to make for example blue 0 0 255 nodata in a RGB
> image even it could be handy because natural aerial and satellite images
> tend to have 0 0 0 and 255 255 255 also in the data pixels but not pure
> blue pixels.
>
>
>
> A good solution would be to polygonise areas with pixel value >=32760 and
> create and add an alpha band or mask band into the image. An easier way is
> to update the raster with gdal_calc
> https://gdal.org/programs/gdal_calc.html
> <https://eur06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgdal.org%2Fprograms%2Fgdal_calc.html&data=05%7C01%7Cjukka.rahkonen%40maanmittauslaitos.fi%7C7fbed8ed633f467be75a08da5898e96a%7Cc4f8a63255804a1c92371d5a571b71fa%7C0%7C0%7C637919711038387709%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=YQ0nkJ%2BQbbXSzsdNj7z6Bj2FOoU7v7KOeQqkeCWKH8E%3D&reserved=0>
> and update all pixel values >=32760 into single value 32767.
>
>
>
> -Jukka Rahkonen-
>
>
>
> *Lähettäjä:* Chao Li <chaoli0394 at gmail.com>
> *Lähetetty:* maanantai 27. kesäkuuta 2022 13.55
> *Vastaanottaja:* Rahkonen Jukka <jukka.rahkonen at maanmittauslaitos.fi>
> *Kopio:* gdal-dev at lists.osgeo.org
> *Aihe:* Re: [gdal-dev] gdalwarp Resampling Input Raster Nodata Setting
>
>
>
> Hi,
>
>
>
> You are right but when I try to check the data processing, there is no any
> compression. On my computer, it is similar to your situation. I guess most
> nodata values are over 32760, so maybe I can list all of them.
>
>
>
> -srcnodata <32760 32761 32762 32763 32764 32765 32766 32767>. Is this the
> right way?
>
>
>
> Thank you.
>
>
>
> Rahkonen Jukka <jukka.rahkonen at maanmittauslaitos.fi> 于2022年6月27日周一 18:14
> 写道：
>
> Hi,
>
>
>
> Please send answers as reply-to-all so they will go also to the gdal-dev
> mailing list.
>
>
>
> There is something odd in the source data. As you can see from the
> attached image there are pixels which are close to the nodata 32767 but not
> exactly. Huge areas have value 32766 but here and there I can also see
> values 32765, 32762 etc. The point in the image is at EPSG:4326
> 97.30902,5.18109.
>
> I think that the source data that you clipped for the test is somehow
> corrupted. Could you check if you can find similar pixel values from the
> original image? The sample is LZW compressed but could it be that at some
> moment the image has been compressed with JPEG or some other lossy method?
>
>
>
> -Jukka Rahkonen-
>
>
>
> *Lähettäjä:* Chao Li <chaoli0394 at gmail.com>
> *Lähetetty:* maanantai 27. kesäkuuta 2022 11.29
> *Vastaanottaja:* Rahkonen Jukka <jukka.rahkonen at maanmittauslaitos.fi>
> *Aihe:* Re: [gdal-dev] gdalwarp Resampling Input Raster Nodata Setting
>
>
>
> Hi,
>
>
>
> Now, seemingly, it works. I am also able to get output raster, but once
> you visualize this raster you will find that the boundaries between value
> and nodata zones are abnormal. There might be over 20,000.
>
>
>
> However the valid values are no more than 1000. That’s my problem.
>
>
>
> Could please check whether you have the same problem?
>
>
>
> Thank you.
>
>
>
> Rahkonen Jukka <jukka.rahkonen at maanmittauslaitos.fi>于2022年6月27日 周一17:17写道：
>
> Hi,
>
>
>
> It seems to work for me with GDAL 3.6.0dev on Windows.
>
>
>
> gdalwarp -t_srs EPSG:4326 -r average -ot float64 -multi
> MOD17A2H_GPP_2020_001.tif mod64m.tif
>
> Creating output file that is 2969P x 2572L.
>
> Processing MOD17A2H_GPP_2020_001.tif [1/1] : 0Using internal nodata values
> (e.g. 32767) for image MOD17A2H_GPP_2020_001.tif.
>
> Copying nodata values from source MOD17A2H_GPP_2020_001.tif to destination
> mod64m.tif.
>
> ...10...20...30...40...50...60...70...80...90...100 - done.
>
>
>
> gdalinfo mod64m.tif
>
> Driver: GTiff/GeoTIFF
>
> Files: mod64m.tif
>
> Size is 2969, 2572
>
> Coordinate System is:
>
> GEOGCRS["WGS 84",
>
> …
> Band 1 Block=2969x1 Type=Float64, ColorInterp=Gray
>
> NoData Value=32767
>
> I don’t know why nodata is lost for you. Have you already tried to set the
> output nodata explicitly with -dstnodata 32767 as a workaround?
>
>
> -Jukka Rahkonen-
>
>
>
> *Lähettäjä:* Chao Li <chaoli0394 at gmail.com>
> *Lähetetty:* maanantai 27. kesäkuuta 2022 10.50
> *Vastaanottaja:* Rahkonen Jukka <jukka.rahkonen at maanmittauslaitos.fi>
> *Aihe:* Re: [gdal-dev] gdalwarp Resampling Input Raster Nodata Setting
>
>
>
> Dear Ms./Mr. Jukka
>
>
>
> Thank you for your reply. I try to delete the -srcnodata 32767, but it
> still doesn't work.
>
>
>
> I have attach a part of raster, in this email.
>
>
>
> Thanks for your help
>
>
>
> Best regards
>
>
>
> Rahkonen Jukka <jukka.rahkonen at maanmittauslaitos.fi> 于2022年6月27日周一 14:52
> 写道：
>
> Hi,
>
>
>
> The right syntax is average -srcnodata 32767 but you should not need to
> use it at all because GDAL recognizes nodata from your input automatically
> as you can see from the gdalinfo report. Could you clip and share a small
> sample from your source data so that some nodata is included? You can use
> gdal_translate with -srswin or -projwin for that.
>
>
>
> -Jukka Rahkonen-
>
>
>
> *Lähettäjä:* gdal-dev <gdal-dev-bounces at lists.osgeo.org> *Puolesta *Chao
> Li
> *Lähetetty:* maanantai 27. kesäkuuta 2022 4.09
> *Vastaanottaja:* gdal-dev at lists.osgeo.org
> *Aihe:* [gdal-dev] gdalwarp Resampling Input Raster Nodata Setting
>
>
>
> Dear GDAL experts,
>
>
>
> I have a problem using *gdalwarp*.
>
>
>
> I want to reproject a raster and change its resolution by the average
> method.
>
>
>
> This is the *gdalinfo* of the raster
>
>
>
> [q70176a at ito-2 ~]$ gdalinfo
> DP12/11_RawData/02_GGP/Gross_PP_8Days_500m_v6/GPP/MOD17A2H_GPP_2016_001.tif
> Driver: GTiff/GeoTIFF
> Files:
> DP12/11_RawData/02_GGP/Gross_PP_8Days_500m_v6/GPP/MOD17A2H_GPP_2016_001.tif
> Size is 86400, 31200
> Coordinate System is:
> PROJCRS["unknown",
> BASEGEOGCRS["unknown",
> DATUM["unknown",
> ELLIPSOID["unknown",6371007.181,0,
> LENGTHUNIT["metre",1,
> ID["EPSG",9001]]]],
> PRIMEM["Greenwich",0,
> ANGLEUNIT["degree",0.0174532925199433,
> ID["EPSG",9122]]]],
> CONVERSION["Sinusoidal",
> METHOD["Sinusoidal"],
> PARAMETER["Longitude of natural origin",0,
> ANGLEUNIT["degree",0.0174532925199433],
> ID["EPSG",8802]],
> PARAMETER["False easting",0,
> LENGTHUNIT["metre",1],
> ID["EPSG",8806]],
> PARAMETER["False northing",0,
> LENGTHUNIT["metre",1],
> ID["EPSG",8807]]],
> CS[Cartesian,2],
> AXIS["easting",east,
> ORDER[1],
> LENGTHUNIT["metre",1,
> ID["EPSG",9001]]],
> AXIS["northing",north,
> ORDER[2],
> LENGTHUNIT["metre",1,
> ID["EPSG",9001]]]]
> Data axis to CRS axis mapping: 1,2
> Origin = (-20015109.353999998420477,7783653.637667000293732)
> Pixel Size = (463.312716527771215,-463.312716527787586)
> Metadata:
> AREA_OR_POINT=Area
> Image Structure Metadata:
> INTERLEAVE=BAND
> Corner Coordinates:
> Upper Left (-20015109.354, 7783653.638) (166d17' 5.25"W, 70d 0' 0.00"N)
> Lower Left (-20015109.354,-6671703.118) ( 0d 0' 0.00"E, 60d 0' 0.00"S)
> Upper Right (20015109.354, 7783653.638) (166d17' 5.25"E, 70d 0' 0.00"N)
> Lower Right (20015109.354,-6671703.118) ( 0d 0' 0.00"W, 60d 0' 0.00"S)
> Center ( -0.000, 555975.260) ( 0d 0' 0.00"W, 5d 0' 0.00"N)
> Band 1 Block=86400x1 Type=Int16, ColorInterp=Gray
> NoData Value=32767
>
>
>
> This is the *gdalwarp* commend:
>
>
>
> [q70176a at ito-2 ~]$ gdalwarp -t_srs EPSG:4326 -te -180 -90 180 90 -tr 0.1
> 0.1 -ot Float64 -r average -srcnodata "32767" -multi -overwrite
> DP12/11_RawData/02_GGP/Gross_PP_8Days_500m_v6/GPP/MOD17A2H_GPP_2016_001.tif
> DP12/11_RawData/02_GGP/Gross_PP_8Days_500m_v6/GPP_01_test/test2.tif
>
>
>
> I also tried
>
>
>
> [q70176a at ito-2 ~]$ gdalwarp -t_srs EPSG:4326 -te -180 -90 180 90 -tr 0.1
> 0.1 -ot Float64 -r average -srcnodata 32767 -multi -overwrite
> DP12/11_RawData/02_GGP/Gross_PP_8Days_500m_v6/GPP/MOD17A2H_GPP_2016_001.tif
> DP12/11_RawData/02_GGP/Gross_PP_8Days_500m_v6/GPP_01_test/test2.tif
>
>
>
> However, the 32767 is not set to be nodata, and be averaged in the output
> raster.
>
>
>
> How can I set this?
>
>
>
> Additionally, I am using a HPC with multinodes. With 8 nodes, I have 288.
> If I set -multi and -wo NUM_THREADS=val/ALL_CPUS, will all that 288 run
> together?
>
>
>
> Since there nothing returns, I really do not know.
>
>
>
> Best regards.
>
>
>
> --
>
>
> ------------------------------------------------------------------------------------
>
> Mike Li
>
> Department of Urban and Environmental Engineering
>
> Graduate School of Engineering
>
> Kyushu University
>
> 744 Motooka, Nishi-ku, Fukuoka 819-0395 Japan
>
> Tel: 090-8304-8953
>
> E-mail: chaoli0394 at gmail.com
>
>
> -------------------------------------------------------------------------------------
>
> 李潮（リ チョウ）
>
> 九州大学 大学院 工学府 都市環境システム工学専攻
>
> 都市工学研究室
>
>
> -------------------------------------------------------------------------------------
>
>
>
>
> --
>
>
> ------------------------------------------------------------------------------------
>
> Mike Li
>
> Department of Urban and Environmental Engineering
>
> Graduate School of Engineering
>
> Kyushu University
>
> 744 Motooka, Nishi-ku, Fukuoka 819-0395 Japan
>
> Tel: 090-8304-8953
>
> E-mail: chaoli0394 at gmail.com
>
>
> -------------------------------------------------------------------------------------
>
> 李潮（リ チョウ）
>
> 九州大学 大学院 工学府 都市環境システム工学専攻
>
> 都市工学研究室
>
>
> -------------------------------------------------------------------------------------
>
> --
>
>
> ------------------------------------------------------------------------------------
>
> Mike Li
>
> Department of Urban and Environmental Engineering
>
> Graduate School of Engineering
>
> Kyushu University
>
> 744 Motooka, Nishi-ku, Fukuoka 819-0395 Japan
>
> Tel: 090-8304-8953
>
> E-mail: chaoli0394 at gmail.com
>
>
> -------------------------------------------------------------------------------------
>
> 李潮（リ チョウ）
>
> 九州大学 大学院 工学府 都市環境システム工学専攻
>
> 都市工学研究室
>
>
> -------------------------------------------------------------------------------------
>
>
>
>
> --
>
>
> ------------------------------------------------------------------------------------
>
> Mike Li
>
> Department of Urban and Environmental Engineering
>
> Graduate School of Engineering
>
> Kyushu University
>
> 744 Motooka, Nishi-ku, Fukuoka 819-0395 Japan
>
> Tel: 090-8304-8953
>
> E-mail: chaoli0394 at gmail.com
>
>
> -------------------------------------------------------------------------------------
>
> 李潮（リ チョウ）
>
> 九州大学 大学院 工学府 都市環境システム工学専攻
>
> 都市工学研究室
>
>
> -------------------------------------------------------------------------------------
>
>
>
>
> --
>
>
> ------------------------------------------------------------------------------------
>
> Mike Li
>
> Department of Urban and Environmental Engineering
>
> Graduate School of Engineering
>
> Kyushu University
>
> 744 Motooka, Nishi-ku, Fukuoka 819-0395 Japan
>
> Tel: 090-8304-8953
>
> E-mail: chaoli0394 at gmail.com
>
>
> -------------------------------------------------------------------------------------
>
> 李潮（リ チョウ）
>
> 九州大学 大学院 工学府 都市環境システム工学専攻
>
> 都市工学研究室
>
>
> -------------------------------------------------------------------------------------
>
--
------------------------------------------------------------------------------------
Mike Li
Department of Urban and Environmental Engineering
Graduate School of Engineering
Kyushu University
744 Motooka, Nishi-ku, Fukuoka 819-0395 Japan
Tel: 090-8304-8953
E-mail: chaoli0394 at gmail.com
-------------------------------------------------------------------------------------
李潮（リ チョウ）
九州大学 大学院 工学府 都市環境システム工学専攻
都市工学研究室
-------------------------------------------------------------------------------------
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20220628/1a859f0a/attachment-0001.htm>
More information about the gdal-dev
mailing list