[gdal-dev] four threads are huh, five are buh? Regressions w/ GDAL 3.5.2/PROJ_9.1.0

[Even Rouault]

Le 12/10/2022 à 08:59, Christian Brock a écrit :
> Hi Even,
>
> Thanks for the fast answer.
>
> We do not share any gdal datasets, BUT we share all spatial references and all coordinate transformations between threads. It used to work in the past, hmm ...
>
> I assume this is a problem?

yes, this was never guaranteed to be safe, even if it was mostly in 
practice before PROJ 6. But with PROJ >= 6, you can't use the same 
SpatialReference or CoordinateTranfsormation concurrently from several 
threads, even for read-only operations, since some internal state might 
be changed.

Even

>
> Cheers --Chr.
>
> -----Original Message-----
> From: Even Rouault <even.rouault at spatialys.com>
>
> Christian,
>
> I suspect the issue is not that much jumping from 4 to 5 threads, but some latent issue in your code or in GDAL/PROJ that becomes more visible. First thing to check: are you sure you are not using a given GDAL / OGR object simultaneously in more than one thread ?
>
> If that's not the issue, if you want effective help, I'm afraid you'll have to go through the effort of writing a minimal standalone reproducer
> (code+data) that you can share with us.
>
> Even
>
> Le 11/10/2022 à 14:56, Christian Brock via gdal-dev a écrit :
>> I have some special issue and would be glad for suggestions.
>>
>> We run complex regression tests using environmental files -- too
>> complex for a bug report ;-(
>>
>> Moving from GDAL_2.4.2/PROJ_5.2.0 to GDAL 3.5.2/PROJ_9.1.0 we run into regressions.
>>
>> The most stunning observation is that the issue only occurs when USING MORE THAN FOUR THREADS!
>> Can anyone think of something that is true with five or more threads but not below?
>>
>> Besides this I should mention, that we use Java bindings
>>
>> The issue is
>> - file format independent,
>> - were observed on ubuntu 20.04 and 22.04,
>> - affects a sample with the following coordinate system:
>> PROJCS["Transverse Mercator",GEOGCS["Potsdam_IST_V2_0",DATUM["Potsdam_IST_V2_0",SPHEROID["Bessel",6377397.155,299.152813106079],TOWGS84[583,68,399.5,0,0,1.36E-05,1.13E-05]],PRIMEM["Greenwich",0],UNIT["degree",0.0174532925199433,AUTHORITY["EPSG","9122"]]],PROJECTION["Transverse_Mercator"],PARAMETER["latitude_of_origin",0],PARAMETER["central_meridian",9],PARAMETER["scale_factor",1],PARAMETER["false_easting",3500000],PARAMETER["false_northing",0],UNIT["metre",1,AUTHORITY["EPSG","9001"]],AXIS["Easting",EAST],AXIS["Northing",NORTH]]"
>> - WGS84 and all UTM systems are good.
>>
>> I should also mention, that this issue affects only the second sample, when I run more than one sample with the PROJCS above.
>> When I change the sample order, it is the second sample again!
>>
>> And as a final point, there are random memory corruptions every couple of hours.
>>
>> So again, please step forward when you know of anything inside GDAL, PROJ or Java bindings that cares about the number of threads.
>>
>> Many thanks
>> --Chr.
>>
>>
>>
>> This email and the information contained herein is proprietary and
>> confidential and subject to the Amdocs Email Terms of Service, which
>> you may review at https://www.amdocs.com/about/email-terms-of-service
>> <https://www.amdocs.com/about/email-terms-of-service>
>>
>> _______________________________________________
>> gdal-dev mailing list
>> gdal-dev at lists.osgeo.org
>> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flist
>> s.osgeo.org%2Fmailman%2Flistinfo%2Fgdal-dev&data=05%7C01%7CChristi
>> an.Brock%40amdocs.com%7C2ed3fa7f029246255d3808daab8a13c3%7Cc8eca3ca127
>> 646d59d9da0f2a028920f%7C0%7C0%7C638010906883663070%7CUnknown%7CTWFpbGZ
>> sb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3
>> D%7C3000%7C%7C%7C&sdata=nOmLEy8X%2FU%2FMlSh1PPvvwRFsYH17H%2F8Fpm17
>> EWPtx5U%3D&reserved=0
> --
> https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.spatialys.com%2F&data=05%7C01%7CChristian.Brock%40amdocs.com%7C2ed3fa7f029246255d3808daab8a13c3%7Cc8eca3ca127646d59d9da0f2a028920f%7C0%7C0%7C638010906883663070%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=2lLYNyBe8NrKkLkZEK7oN6MXq%2Bd7KQMDilS17fVL7Ck%3D&reserved=0
> My software is free, but my time generally not.
>
> This email and the information contained herein is proprietary and confidential and subject to the Amdocs Email Terms of Service, which you may review at https://www.amdocs.com/about/email-terms-of-service <https://www.amdocs.com/about/email-terms-of-service>
>
-- 
http://www.spatialys.com
My software is free, but my time generally not.