[gdal-dev] Thread safety of GDAL projection machinery?

[Even Rouault]

Le Thursday 04 June 2009 21:51:36 Karney, Charles, vous avez écrit :
> I'm afraid FWTools 2.4.0 has the same problem.  Here's a sampling of the
> output from multiple runs extracting the geodata from about 200 geotiffs
> (Florida North projection) running with 6 threads.

Too bad... I'm afraid that at that point we now really need code demonstrating 
the bug to fix it.

Here's a sample code I've used to try to reproduce (without success) the 
issue :

#include <gdal.h>
#include <cpl_multiproc.h>

void thread(void* unused)
{
    CPLSleep(2); /* Dummy way of waiting for all threads being started */
    while(TRUE)
    {
        GDALDatasetH hDS = GDALOpen("foo.tif", GA_ReadOnly);
        const char* pszPrj = GDALGetProjectionRef(hDS);
        GDALClose(hDS);
    }
}

int main(int argc, char* argv[])
{
    GDALAllRegister();
    int i;
    for(i=0;i<100;i++)
    {
        CPLCreateThread(thread, NULL);
    }
    while(TRUE) CPLSleep(1);
    return 0;
}

You could use that as a skeleton and add the GDAL calls that cause the errors. 
The exact image(s) you use could be important too (I don't remember you've 
precised which image format/driveer you use).

>
> ERROR 3: Read of file c:/Program Files (x86)/FWTools2.4.0/data/pcs.csv
> failed. ERROR 1: No PROJ.4 translation for source SRS, coordinate
> transformation initialization has failed. ERROR 3: Read of file c:/Program
> Files (x86)/FWTools2.4.0/data/prime_meridian.csv failed. ERROR 3: Read of
> file c:/Program Files (x86)/FWTools2.4.0/data/pcs.override.csv failed.
> ERROR 3: Read of file c:/Program Files
> (x86)/FWTools2.4.0/data/projop_wparm.csv failed. ERROR 3: Read of file
> c:/Program Files (x86)/FWTools2.4.0/data/gcs.csv failed. ERROR 2:
> CPLRealloc(): Out of memory allocating 32 bytes.
> ERROR 2: CPLRealloc(): Out of memory allocating 12 bytes.
> ERROR 3: Read of file c:/Program Files
> (x86)/FWTools2.4.0/data/unit_of_measure.csv failed.
>
> The problems go away if
>
> (1) I run a single thread, or
> (2) I put a lock around the GDAL calls.

Solution (2) is probably not better than (1) in term of speed, as it is going 
to serialize all GDAL calls.

>
>
> --
> Charles Karney <ckarney at sarnoff.com>
> Sarnoff Corporation, Princeton, NJ 08543-5300
>
> Tel: +1 609 734 2312
> Fax: +1 609 734 2662