<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div class="elementToProof" style="text-align: left; text-indent: 0px; background-color: rgb(255, 255, 255); margin: 0px; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: black;">
Hi Daniel,</div>
<div style="text-align: left; text-indent: 0px; background-color: rgb(255, 255, 255); margin: 0px; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: black;">
<br>
</div>
<div style="text-align: left; text-indent: 0px; background-color: rgb(255, 255, 255); margin: 0px; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: black;">
Thank you. I tried your commands exactly with gdal 3.8.3 and am seeing the odd type of output I was describing in the northeastern most reaches of the images. Would you be willing to verify if you see that or not?</div>
<div style="text-align: left; text-indent: 0px; background-color: rgb(255, 255, 255); margin: 0px; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: black;">
<br>
</div>
<div class="elementToProof" style="text-align: left; text-indent: 0px; background-color: rgb(255, 255, 255); margin: 0px; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: black;">
Here is the result of the debug messaging. I think a screenshot which shows what I see results in the message being over the size limit.</div>
<div style="text-align: left; text-indent: 0px; background-color: rgb(255, 255, 255); margin: 0px; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: black;">
<br>
</div>
<div style="text-align: left; text-indent: 0px; background-color: rgb(255, 255, 255); margin: 0px; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: black;">
GDAL: GDALOpen(../manyPts4000_SRTM_test_NptRPC.tiff, this=0x5584b5c21370) succeeds as GTiff.</div>
<div style="text-align: left; text-indent: 0px; background-color: rgb(255, 255, 255); margin: 0px; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: black;">
GDAL: GDALOpen(output_gdal38.tiff, this=0x5584b5c4c950) succeeds as GTiff.</div>
<div style="text-align: left; text-indent: 0px; background-color: rgb(255, 255, 255); margin: 0px; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: black;">
GDALWARP: Defining SKIP_NOSOURCE=YES</div>
<div style="text-align: left; text-indent: 0px; background-color: rgb(255, 255, 255); margin: 0px; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: black;">
Processing ../manyPts4000_SRTM_test_NptRPC.tiff [1/1] : 0WARP: Copying metadata from first source to destination dataset</div>
<div style="text-align: left; text-indent: 0px; background-color: rgb(255, 255, 255); margin: 0px; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: black;">
WARP: Set SOURCE_EXTRA=5 warping options due to RPC warping</div>
<div style="text-align: left; text-indent: 0px; background-color: rgb(255, 255, 255); margin: 0px; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: black;">
WARP: Set SAMPLE_STEPS=ALL warping options due to RPC DEM warping</div>
<div style="text-align: left; text-indent: 0px; background-color: rgb(255, 255, 255); margin: 0px; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: black;">
GDAL: GDALOpen(srtm_12_05.tif, this=0x5584b5c522a0) succeeds as GTiff.</div>
<div style="text-align: left; text-indent: 0px; background-color: rgb(255, 255, 255); margin: 0px; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: black;">
RPC: Short-circuiting coordinate transformation from DEM SRS to WGS 84 due to apparent nop</div>
<div style="text-align: left; text-indent: 0px; background-color: rgb(255, 255, 255); margin: 0px; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: black;">
GDAL: GDAL_CACHEMAX = 1504 MB</div>
<div style="text-align: left; text-indent: 0px; background-color: rgb(255, 255, 255); margin: 0px; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: black;">
GTiff: ScanDirectories()</div>
<div style="text-align: left; text-indent: 0px; background-color: rgb(255, 255, 255); margin: 0px; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: black;">
GDAL: GDALDefaultOverviews::OverviewScan()</div>
<div style="text-align: left; text-indent: 0px; background-color: rgb(255, 255, 255); margin: 0px; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: black;">
Using internal nodata values (e.g. 0) for image ../manyPts4000_SRTM_test_NptRPC.tiff.</div>
<div style="text-align: left; text-indent: 0px; background-color: rgb(255, 255, 255); margin: 0px; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: black;">
WARP: band=0 dstNoData=0.000000</div>
<div style="text-align: left; text-indent: 0px; background-color: rgb(255, 255, 255); margin: 0px; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: black;">
RPC: Failed Iterations 101: Got: -121.6743548438707,37.81305392714289 Offset=0.200509,-7.35862</div>
<div style="text-align: left; text-indent: 0px; background-color: rgb(255, 255, 255); margin: 0px; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: black;">
WARP: At least one point failed after direct transform</div>
<div style="text-align: left; text-indent: 0px; background-color: rgb(255, 255, 255); margin: 0px; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: black;">
RPC: Failed Iterations 101: Got: -121.6743548438707,37.81305392714289 Offset=0.200509,-7.35862</div>
<div style="text-align: left; text-indent: 0px; background-color: rgb(255, 255, 255); margin: 0px; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: black;">
RPC: Failed Iterations 101: Got: -121.6756957607436,37.81515846637773 Offset=-0.0317686,-12.1573</div>
<div style="text-align: left; text-indent: 0px; background-color: rgb(255, 255, 255); margin: 0px; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: black;">
RPC: Failed Iterations 101: Got: -121.6759688314497,37.81397923057818 Offset=-0.0782949,-21.942</div>
<div style="text-align: left; text-indent: 0px; background-color: rgb(255, 255, 255); margin: 0px; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: black;">
RPC: Failed Iterations 101: Got: -121.6760567085541,37.81393132082147 Offset=-0.0200723,-13.2144</div>
<div style="text-align: left; text-indent: 0px; background-color: rgb(255, 255, 255); margin: 0px; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: black;">
RPC: Failed Iterations 101: Got: -121.6758750148136,37.8150256294207 Offset=0.0121725,-8.26664</div>
<div style="text-align: left; text-indent: 0px; background-color: rgb(255, 255, 255); margin: 0px; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: black;">
RPC: Failed Iterations 101: Got: -121.6758055299304,37.81513471897441 Offset=0.0111142,-9.29081</div>
<div style="text-align: left; text-indent: 0px; background-color: rgb(255, 255, 255); margin: 0px; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: black;">
RPC: Failed Iterations 101: Got: -121.6757471773221,37.81517816256341 Offset=0.00786933,-9.66179</div>
<div style="text-align: left; text-indent: 0px; background-color: rgb(255, 255, 255); margin: 0px; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: black;">
RPC: Failed Iterations 101: Got: -121.6757610763044,37.81516845364766 Offset=0.00852169,-9.58453</div>
<div style="text-align: left; text-indent: 0px; background-color: rgb(255, 255, 255); margin: 0px; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: black;">
GDAL: GDALSuggestedWarpOutput(): 1 out of 3249 points failed to transform.</div>
<div style="text-align: left; text-indent: 0px; background-color: rgb(255, 255, 255); margin: 0px; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: black;">
GDAL: GDALWarpKernel()::GWKNearestShort() Src=0,0,2383x2800 Dst=0,0,2411x3318</div>
<div style="text-align: left; text-indent: 0px; background-color: rgb(255, 255, 255); margin: 0px; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: black;">
...10...20...30...40...50GDAL: GDALWarpKernel()::GWKNearestShort() Src=1710,0,1460x1401 Dst=2411,0,1205x1659</div>
<div style="text-align: left; text-indent: 0px; background-color: rgb(255, 255, 255); margin: 0px; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: black;">
...60GDAL: GDALWarpKernel()::GWKNearestShort() Src=2833,0,1263x2458 Dst=3616,0,1206x829</div>
<div style="text-align: left; text-indent: 0px; background-color: rgb(255, 255, 255); margin: 0px; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: black;">
...GDAL: GDALWarpKernel()::GWKNearestShort() Src=2995,148,1101x1054 Dst=3616,829,1206x830</div>
<div style="text-align: left; text-indent: 0px; background-color: rgb(255, 255, 255); margin: 0px; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: black;">
70..GDAL: GDALWarpKernel()::GWKNearestShort() Src=2036,993,2060x1807 Dst=2411,1659,2411x1659</div>
<div style="text-align: left; text-indent: 0px; background-color: rgb(255, 255, 255); margin: 0px; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: black;">
.80...90...100 - done.</div>
<div style="text-align: left; text-indent: 0px; background-color: rgb(255, 255, 255); margin: 0px; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: black;">
GDAL: GDALClose(srtm_12_05.tif, this=0x5584b5c522a0)</div>
<div style="text-align: left; text-indent: 0px; background-color: rgb(255, 255, 255); margin: 0px; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: black;">
GDAL: Flushing dirty blocks: 0...10...20...30...40...50...60...70...80...90...100 - done.</div>
<div style="text-align: left; text-indent: 0px; background-color: rgb(255, 255, 255); margin: 0px; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: black;">
GDAL: GDALClose(output_gdal38.tiff, this=0x5584b5c4c950)</div>
<div class="elementToProof" style="text-align: left; text-indent: 0px; background-color: rgb(255, 255, 255); margin: 0px; font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: black;">
GDAL: GDALClose(../manyPts4000_SRTM_test_NptRPC.tiff, this=0x5584b5c21370)</div>
<div id="appendonsend"></div>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> Daniel Evans <daniel.fred.evans@gmail.com><br>
<b>Sent:</b> Thursday, April 18, 2024 9:03 AM<br>
<b>To:</b> Joseph McGlinchy <jmcglinchy@hydrosat.com><br>
<b>Cc:</b> Even Rouault <even.rouault@spatialys.com>; gdal-dev@lists.osgeo.org <gdal-dev@lists.osgeo.org><br>
<b>Subject:</b> Re: [gdal-dev] gdalwarp with RPCs causing bogus output</font>
<div> </div>
</div>
<div>
<div dir="ltr">Hi Joe,
<div><br>
</div>
<div>My minimal change is just to add the missing "RPC_DEM=srtm_12_05.tif" and remove the "RPC_HEIGHT=350". RPC_HEIGHT appears to override the use of any supplied DEM, and the docs at least hint that this is the expected behaviour.</div>
<div><br>
</div>
<div>```</div>
<div>gdalwarp --config CPL_DEBUG ON \<br>
-t_srs EPSG:32610 \<br>
-rpc \<br>
-to "RPC_DEM_SRS=+proj=longlat +datum=WGS84 +no_def" \<br>
-to "RPC_DEM_MISSING_VALUE=0.001" \<br>
-to "RPC_FOOTPRINT='POLYGON ((-123.12941327726458 38.01872167010621, -121.67068342817124 37.82615456858018, -121.87575337280391 37.11164114368971, -123.32135086275996 37.30124603928108, -123.12941327726458 38.01872167010621))'" \<br>
-to "RPC_MAX_ITERATIONS=101" \<br>
-to "RPC_DEM=srtm_12_05.tif" \<br>
manyPts4000_SRTM_test_NptRPC.tiff \<br>
output.tiff<br>
</div>
<div>```</div>
<div><br>
</div>
<div>Changing to "RPC_DEM_SRS=EPSG:4326+5773" as Even suggested gives a noticeable change around the DEM locations of dams and lakes with the updated CRS, e.g. west and north of Livermore CA. Whether that's an improvement, or an undesired change due to the
DEM being treated differently between generating the RPCs and using them, I'm not sure!</div>
<div><br>
</div>
<div>Daniel</div>
</div>
<br>
<div class="x_gmail_quote">
<div dir="ltr" class="x_gmail_attr">On Thu, 18 Apr 2024 at 15:39, Joseph McGlinchy <<a href="mailto:jmcglinchy@hydrosat.com">jmcglinchy@hydrosat.com</a>> wrote:<br>
</div>
<blockquote class="x_gmail_quote" style="margin:0px 0px 0px 0.8ex; border-left:1px solid rgb(204,204,204); padding-left:1ex">
<div class="x_msg7550246628656371142">
<div dir="ltr">
<div style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Hi Daniel,</div>
<div style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Thanks for trying with the data! I appreciate that. The gdal command I provided in the sample-gdal-call.txt file is the only one I could get to have "good" looking results. What command did you use to include the DEM?</div>
<div style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
If there has been a fix between 3.6 and 3.8.3 that helps with this, then I'm happy to try it. I'll give that a look too.</div>
<div style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
thanks!</div>
<div style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Joe</div>
<div id="x_m_7550246628656371142appendonsend"></div>
<hr style="display:inline-block; width:98%">
<div id="x_m_7550246628656371142divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" color="#000000" style="font-size:11pt"><b>From:</b> Daniel Evans <<a href="mailto:daniel.fred.evans@gmail.com" target="_blank">daniel.fred.evans@gmail.com</a>><br>
<b>Sent:</b> Thursday, April 18, 2024 5:44 AM<br>
<b>To:</b> Joseph McGlinchy <<a href="mailto:jmcglinchy@hydrosat.com" target="_blank">jmcglinchy@hydrosat.com</a>><br>
<b>Cc:</b> Even Rouault <<a href="mailto:even.rouault@spatialys.com" target="_blank">even.rouault@spatialys.com</a>>;
<a href="mailto:gdal-dev@lists.osgeo.org" target="_blank">gdal-dev@lists.osgeo.org</a> <<a href="mailto:gdal-dev@lists.osgeo.org" target="_blank">gdal-dev@lists.osgeo.org</a>><br>
<b>Subject:</b> Re: [gdal-dev] gdalwarp with RPCs causing bogus output</font>
<div> </div>
</div>
<div>
<div dir="ltr">Hi Joe,
<div><br>
</div>
<div>Trying your commands with GDAL 3.8.3, the results don't seem to be as bad as what you're reporting - I don't see big strips missing. Adding in the DEM file as Even mentioned, it seems to be doing what I'd expect (the correction isn't "right" because of
the georeferencing offset, but it's doing what it's told using the underlying DEM).</div>
<div><br>
</div>
<div>Any possibility of trying a newer GDAL version in case there has been a fix since 3.6.0?</div>
<div><br>
</div>
<div>Cheers,</div>
<div>Daniel</div>
</div>
<br>
<div>
<div dir="ltr">On Thu, 18 Apr 2024 at 02:35, Joseph McGlinchy via gdal-dev <<a href="mailto:gdal-dev@lists.osgeo.org" target="_blank">gdal-dev@lists.osgeo.org</a>> wrote:<br>
</div>
<blockquote style="margin:0px 0px 0px 0.8ex; border-left:1px solid rgb(204,204,204); padding-left:1ex">
<div>
<div dir="ltr">
<div style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Hi Even,</div>
<div style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Thank you for this great response. I'll try out some of those things you mention on the CRS of the DEM and correcting for the geoid; I had also been reading that any height values used in this process need to be heights above the ellipsoid. </div>
<div style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
I had seen good outputs from <code>gdaltransform</code>, both forward and inverse, and can get back to the corner coordinates, center coordinates, etc. in both lon/lat and sample/line so it could be a subtlety of the DEM + CRS, or the more challenging issue
of stable RPCs. </div>
<div style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
On the topic of stable RPCs, do you happen to know if it is enough to zero out the higher-order terms after the full set of coefficients are fit? Since there are squared terms in the higher-order coefficients, that could definitely be part of the issue, even
if it works mathematically.</div>
<div style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
thanks!</div>
<div style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Joe</div>
<div id="x_m_7550246628656371142x_m_282046413381828842appendonsend"></div>
<hr style="display:inline-block; width:98%">
<div id="x_m_7550246628656371142x_m_282046413381828842divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" color="#000000" style="font-size:11pt"><b>From:</b> Even Rouault <<a href="mailto:even.rouault@spatialys.com" target="_blank">even.rouault@spatialys.com</a>><br>
<b>Sent:</b> Wednesday, April 17, 2024 4:20 PM<br>
<b>To:</b> Joseph McGlinchy <<a href="mailto:jmcglinchy@hydrosat.com" target="_blank">jmcglinchy@hydrosat.com</a>>;
<a href="mailto:gdal-dev@lists.osgeo.org" target="_blank">gdal-dev@lists.osgeo.org</a> <<a href="mailto:gdal-dev@lists.osgeo.org" target="_blank">gdal-dev@lists.osgeo.org</a>><br>
<b>Subject:</b> Re: [gdal-dev] gdalwarp with RPCs causing bogus output</font>
<div> </div>
</div>
<div>
<p>Hi,</p>
<p><br>
</p>
<p>I haven't done myself that exercice but I know that computing RPC values that are stable enough might be challenging, so perhaps the issue is related to that<br>
</p>
<p><br>
</p>
<p>A few other remarks:</p>
<p>- your gdalwarp command line refers to RPC_DEM_SRS and RPC_DEM_MISSING_VALUE but doesn't include a RPC_DEM itself, hence those are likely to be non effective</p>
<p>- you generally don't want to use both RPC_HEIGHT and RPC_DEM. RPC_HEIGHT is essentially useful when you don't have a DEM available, and thus fallback taking an average elevation</p>
<p>- there's a subtlety regarding DEM. RPC reference for altitudes is WGS84 ellipsoidal height, not orthometric/MSL altitude. But DEM values use orthometric/MSL altitude, hence a geoid correction must be applied. So you'd rather want to use RPC_DEM_SRS=EPSG:4326+5773
for example to use the EGM96 geoid (cf <a href="https://github.com/OSGeo/gdal/issues/3298" target="_blank">
https://github.com/OSGeo/gdal/issues/3298</a>). That said, misusing orthometric altitude vs ellipsoidal height generally accounts for small shifts, not big ones</p>
<p>- you could use gdaltransform with all your -to parameter and input_with_RPCs.tiff to check at least that the forward RPC transformation path works correctly (the one from longitude, latitude -> column, line). And with -i to check the inverse RPC transformer.
The inverse RPC transformer can have a hard time converging in montainous areas or for images off-nadir</p>
<p><br>
</p>
<p>Even<br>
</p>
<p><br>
</p>
<div>Le 17/04/2024 à 23:30, Joseph McGlinchy via gdal-dev a écrit :<br>
</div>
<blockquote type="cite">
<div style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Hello,</div>
<div style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
I am attempting to implement georegistration through RPC. I have the following information I've used to calibrate the RPC coefficients, using all terms for numerator and denominator for both sample and line equations.</div>
<div style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<ul style="margin-top:0px; margin-bottom:0px; list-style-type:disc">
<li style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<div style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
image grid, stored in tiff format with no geo-information associated with it, so it reflects the imaging orientation</div>
</li><li style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<div style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
a 'ground' grid, which corresponds to the longitude/latitude coordinates for each pixel determined from a line-of-sight vector and imaging system coordinates where the pseudo-rays intersect the WGS84 geoid</div>
</li><li style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<div style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
a random sample of up to 4000 image coordinate / object space coordinate pairs (I arrived at this number through trial and error; using all pixels explodes RAM)</div>
</li><li style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<div style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
elevation extracted at the longitude/latitude coordinates from a DEM, in this case, SRTM, but have also tried using NASADEM</div>
</li></ul>
<div style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
I am able to write out an image to EPSG:4326 by populating the RPC metadata and using the non-georeferenced image data. However, I am struggling to use
<code>gdalwarp</code> in a reliable way to orthorectify that data, let alone writing it out in a way that 'bakes in' the georegistration with the RPCs whether that is in EPSG:4326 or the local UTM zone. I see strips of the image "ripped out" with odd curves
in various places throughout the image. </div>
<div style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
The only way I've been able to use <code>gdalwarp</code> to write the image at all is with the following parameters (any DEM reference is to the SRTM DEM):<br>
<br>
</div>
<div style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
gdalwarp --config CPL_DEBUG ON -t_srs EPSG:32610 -rpc -to "RPC_DEM_SRS=+proj=longlat +datum=WGS84 +no_def" -to "RPC_HEIGHT=350" -to "RPC_DEM_MISSING_VALUE=0.001" -to "RPC_FOOTPRINT='POLYGON ((list of polygon coordinates comprising the long/lat grid))'" -to
"RPC_MAX_ITERATIONS=101" input_with_RPCs.tiff output.tiff</div>
<div style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
This is the only configuration i can use to run <code>gdalwarp</code> successfully. removing any single RPC_X tranformer option gives me bogus output. The RPC_HEIGHT value i specify above is not close to the mean or median elevation of the extent of my data;
mean is ~195m and median is ~150m.</div>
<div style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
With the debug turned on, any other set of parameters gives me failed RPC convergence on several points. I am able to reproduce this regularly by specifying RPC_DEM=dem.tif, where dem.tif is the same data I used to extract elevation values when calibrating
the RPCs. I am seeing normalized latitude and longitude values with magnitude > 1 (I checked every location in the image, based on the metadata, the range is not outside of [-1,1]), as well as normalized altitude values with magnitude > 1 (there are some,
not many, that have magnitude of 1.75). </div>
<div style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
My workflow can be summarized as:<br>
<br>
</div>
<ol start="1" style="margin-top:0px; margin-bottom:0px; list-style-type:decimal">
<li style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<div style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
load grids (image data, longitude, latitude)</div>
</li><li style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:16px; color:rgb(0,0,0)">
<div style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:16px; color:rgb(0,0,0)">
<span style="background-color:rgb(255,255,255)">randomly sample up to 4000 points in image coordinates, object coordinates</span></div>
</li><ol start="1" style="margin-top:0px; margin-bottom:0px; list-style-type:lower-alpha">
<li style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:16px; color:rgb(0,0,0)">
<div style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:16px; color:rgb(0,0,0)">
<span style="background-color:rgb(255,255,255)">assign z-value from SRTM DEM</span></div>
</li><li style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:16px; color:rgb(0,0,0)">
<div style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:16px; color:rgb(0,0,0)">
<span style="background-color:rgb(255,255,255)">evaluate if any of the points are in NODATA areas of SRTM (image is coastal, so there are NODATA areas for SRTM here), if so, remove those and generate more points</span></div>
</li></ol>
<li style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<div style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
normalize coordinates of grids to be in [-1,1], recording offsets and scale</div>
</li><li style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<div style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
calibrate RPC coefficients using all terms</div>
</li><li style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<div style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
write out GeoTIFF with image grid for pixels, along with RPC required metadata fields and CRS EPSG:4326</div>
</li></ol>
<div style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
System information (please let me know if more is needed)</div>
<div style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
OS: Ubuntu 20.04 LTS (GNU/Linux 5.10.16.3-microsoft-standard-WSL2 x86_64) (Windows Subsystem for Linux)</div>
<div style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
GDAL 3.6.0 (python)</div>
<div style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Thank you in advance for any insight into this process! I am happy to package up any of the data I am using, as well. I have placed the initial data from the end of Step 5 described above, along with some additional files, a sample gdalwarp call, and a file-list.txt,
at <a href="https://drive.google.com/drive/folders/1BfevhKQa4ZHi_OQfiX_rUk2sqeoNVhyM?usp=sharing" id="x_m_7550246628656371142x_m_282046413381828842LPlnkOWAef49c06e-b1a7-175b-d89e-e930398f4b9a" target="_blank">
https://drive.google.com/drive/folders/1BfevhKQa4ZHi_OQfiX_rUk2sqeoNVhyM?usp=sharing</a></div>
<div style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
If more is needed for anyone interested in having a look, please let me know and I'll upload.</div>
<div style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Cheers,</div>
<div style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
Joe</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<br>
<fieldset></fieldset>
<pre>_______________________________________________
gdal-dev mailing list
<a href="mailto:gdal-dev@lists.osgeo.org" target="_blank">gdal-dev@lists.osgeo.org</a>
<a href="https://lists.osgeo.org/mailman/listinfo/gdal-dev" target="_blank">https://lists.osgeo.org/mailman/listinfo/gdal-dev</a>
</pre>
</blockquote>
<pre cols="72">--
<a href="http://www.spatialys.com" target="_blank">http://www.spatialys.com</a>
My software is free, but my time generally not.</pre>
</div>
</div>
_______________________________________________<br>
gdal-dev mailing list<br>
<a href="mailto:gdal-dev@lists.osgeo.org" target="_blank">gdal-dev@lists.osgeo.org</a><br>
<a href="https://lists.osgeo.org/mailman/listinfo/gdal-dev" rel="noreferrer" target="_blank">https://lists.osgeo.org/mailman/listinfo/gdal-dev</a><br>
</div>
</blockquote>
</div>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</body>
</html>