[Gdal-dev] Missing pixels after warp

[Frank Warmerdam]

Stephen Woodbridge wrote:
> Thank you for the offer. I was able to track the problem back to the 
> source tiles and it is a nearblack issue with some tiles.

Steve,

Ah, I didn't know nearblack was involved, or I wouldn't have been so
surprised by the effect.


> This brings up couple of interesting problems associated with using 
> nearblack. The imagery I'm working with is CCM files and as you know 
> these are mosaics of DOQQ or equivalent tiles. So the one example that I 
> traced back had an image with a notch like where "n" is nearblack:
> 
>         nnnnnnnnnnnnn          nnnnnnnnnnnnnnn
>         n+---------+n          n+-----------+n
>         n|         |n     A    n|           |n
>         n|         |n          n|           |n
>         n|         |nnnnnnnnnnnn|           |nnnnnnnnnnnnnnn
>         n|         +------------+           +-------------+n
>         n|                                                |n
>         n|                                                |n
>         n+------------------------------------------------+n
>         nnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnn
> 
> 1) nearblack scans rows from right to left and left to right and stops 
> its scan after eating some pixels of good data. So it will never process 
> a notch like area "A" above or a hole in the imagery (not that I have 
> that case).

Yes, this is a known problem.  The difficulty is that it is noticably more
complicated to do the top to bottom, and bottom to top scanning without using
a lot of memory, though I can imagine some approaches that wouldn't be bad.

> 2) Notice my comment about eating some good pixels. nearblack does not 
> implement a look ahead, it destructively eats (sets to black) good 
> pixels and counts them. After it has reached some threshold it decides 
> that it is in real data and terminates that scan. What is should do is 
> trigger a look ahead if it finds a pixel outside the near black 
> threshhold, to see if it is a small outlier or real data. Yes this would 
> make it marginally slower but most of the time you would not do the look 
> ahead, and the current algorithm does the same scan as a look behind.

This is a reasonable idea, though in my clients case it was more important
to ensure that the edges were eaten away cleanly (since there was lots of
overlap) than to preserve the maximum number of pixels.

> 3) it would be a nice feature to add to gdalwarp and gdal_translate to 
> be able to define something like:
>    -srcnodata <5
> that would treat all values less than 5 as nodata in the source image. 
> This has some obvious downsides of setting possible nearblack good data 
> in the interior to black, but I think it would be generally useful for a 
> lot of situations.

I'm a bit dubious about this approach.  I think if you really want more
generalized concepts of nodata values and regions, the thing to do is
preprocess your image and marking nodata pixels explicitly.  This could be
done in something grass which has excellent "raster algebra" tools, or using
something like a python script with numpy to operate on the image as an array.

Best regards,
-- 
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up   | Frank Warmerdam, warmerdam at pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush    | President OSGeo, http://osgeo.org