[mapguide-users] MrSID files through GDAL FDO provider

[Kenneth, GEOGRAF A/S]

I mean no disrespect. I have great respect for your work.
I was merely trying to point out a optimization posibility for your 
library, when used with MapGuide.

I am not refering to the way a provider loads its data, as that is 
probably fine and very fast.
I am refering to the folowing code (from gdaldataset.cpp):

    for( iDriver = 0; iDriver < poDM->GetDriverCount(); iDriver++ )
    {
        GDALDriver      *poDriver = poDM->GetDriver( iDriver );
        GDALDataset     *poDS;

        poDS = poDriver->pfnOpen( &oOpenInfo );
        if( poDS != NULL )
        {
            poDS->SetDescription( pszFilename );

            if( poDS->poDriver == NULL )
                poDS->poDriver = poDriver;

           
            CPLDebug( "GDAL", "GDALOpen(%s) succeeds as %s.",
                      pszFilename, poDriver->GetDescription() );

            return (GDALDatasetH) poDS;
        }

        if( CPLGetLastErrorNo() != 0 )
            return NULL;
    }

This code tries each provider in turn, in an attempt to find a suitable one.
 From the user descriptions I have seen, this leads to some providers 
trying to read much of the file, and then fail.
This is fine, and it moves on to the next. But since the files can be 
very large, it might be a good idea to adjust the order of testing.
That was my suggestion. While finding the above code, i stumbled upon 
this code:

void GDALDriverManager::AutoSkipDrivers()

{
    if( CPLGetConfigOption( "GDAL_SKIP", NULL ) == NULL )
        return;

    char **papszList = CSLTokenizeString( 
CPLGetConfigOption("GDAL_SKIP","") );

    for( int i = 0; i < CSLCount(papszList); i++ )
    {
        GDALDriver *poDriver = GetDriverByName( papszList[i] );

        if( poDriver == NULL )
            CPLError( CE_Warning, CPLE_AppDefined,
                      "Unable to find driver %s to unload from GDAL_SKIP 
environment variable.",
                      papszList[i] );
        else
        {
            CPLDebug( "GDAL", "AutoSkipDriver(%s)", papszList[i] );
            DeregisterDriver( poDriver );
            delete poDriver;
        }
    }

    CSLDestroy( papszList );
}

Which does sort of what I was proposing, but in a better way, since 
there is no file overhead.
So, to skip certain providers, declare an environment variable with the 
name GDAL_SKIP, and enter in the names of the providers to skip.

Regards, Kenneth, GEOGRAF A/S



Frank Warmerdam skrev:
> Kenneth, GEOGRAF A/S wrote:
>> The current problem is the Open() call, which tests the providers in 
>> turn.
>> If there was a file in the folder (say "GDAL Loader Hints.txt" 
>> stating ea.:
>> MrSID
>> ECW
>
> Kenneth,
>
> You state that the "current problem is the Open() call, which tests the
> providers in turn.".  What basis do you have to think this is a problem?
>
> I would stress that this is the same mechanism GDAL has used for eight
> years, and has been the basis of the raster rendering in MapServer which
> is widely accepted as "fast".
>
> For instance, this is the first code in ECWDataset::Open():
>
>     if( EQUALN(poOpenInfo->pszFilename,"J2K_SUBFILE:",12) )
>         return Open( poOpenInfo );
>
>     else if( poOpenInfo->nHeaderBytes >= 16
>         && (memcmp( poOpenInfo->pabyHeader, jpc_header,
>                     sizeof(jpc_header) ) == 0
>             || memcmp( poOpenInfo->pabyHeader, jp2_header,
>                     sizeof(jp2_header) ) == 0) )
>         return Open( poOpenInfo );
>
>     else
>         return NULL;
>
> Basically, a string comparison, and then one or two comparisons of
> a few bytes of the file header (pre-read and reused for each driver).
> This is *fast*.  There are *real* opportunities for optimization out
> there.  This is not one of them.
>
> OK, rather than drag this out, I'm going to stop responding to this
> thread.
>
> Best regards,