[gdal-dev] There's something strange about two tests that are very similar
Even Rouault
even.rouault at spatialys.com
Thu Mar 14 02:12:59 PDT 2024
gdb is not going to help for test_ogr_basic_10. As you can see in the
test, it "runexternal", that is it forks a child process, so gdb isn't
going to debug this child process by default.
The easiest is to run "test_ogrsf -all_drivers" directly / under gdb /
under valgrind
Actually, I've just tried that on my system. Nothing under Valgrind, but
the execution time was quite slow once it reaches the Miramon driver,
and breaking under gdb, I see the following
INFO: MiraMonVector: Creation of /foo/test.pol should have failed.
INFO: MiraMonVector: CreateFeature() at line 780 succeeded but failed to
assign FID to feature.
^C--Type <RET> for more, q to quit, c to continue without paging--
Thread 1 "test_ogrsf" received signal SIGINT, Interrupt.
__memset_avx2_erms () at
../sysdeps/x86_64/multiarch/memset-vec-unaligned-erms.S:151
151 ../sysdeps/x86_64/multiarch/memset-vec-unaligned-erms.S: Aucun
fichier ou dossier de ce type.
(gdb) bt
#0 __memset_avx2_erms () at
../sysdeps/x86_64/multiarch/memset-vec-unaligned-erms.S:151
#1 0x00007ffff5e40ed5 in CPLCalloc (nCount=1, nSize=104857600) at
/home/even/gdal/MiraMon-Vector-driver/port/cpl_conv.cpp:164
#2 0x00007ffff6f39a33 in MMMoveFromFileToFile (pSrcFile=0x555555bcd870,
pDestFile=0x5555562b3b90, nOffset=0x5555561f3990)
at
/home/even/gdal/MiraMon-Vector-driver/ogr/ogrsf_frmts/miramon/mm_wrlayr.c:2693
#3 0x00007ffff6f388fb in MMCloseArcLayer (hMiraMonLayer=0x5555561e4f90)
at
/home/even/gdal/MiraMon-Vector-driver/ogr/ogrsf_frmts/miramon/mm_wrlayr.c:2150
#4 0x00007ffff6f38dc5 in MMCloseLayer (hMiraMonLayer=0x5555561e4f90) at
/home/even/gdal/MiraMon-Vector-driver/ogr/ogrsf_frmts/miramon/mm_wrlayr.c:2283
#5 0x00007ffff6f2d955 in OGRMiraMonLayer::~OGRMiraMonLayer
(this=0x5555561d5aa0, __in_chrg=<optimized out>) at
/home/even/gdal/MiraMon-Vector-driver/ogr/ogrsf_frmts/miramon/ogrmiramonlayer.cpp:535
#6 0x00007ffff6f2de22 in OGRMiraMonLayer::~OGRMiraMonLayer
(this=0x5555561d5aa0, __in_chrg=<optimized out>) at
/home/even/gdal/MiraMon-Vector-driver/ogr/ogrsf_frmts/miramon/ogrmiramonlayer.cpp:619
#7 0x00007ffff6f2af7a in OGRMiraMonDataSource::~OGRMiraMonDataSource
(this=0x555556265cb0, __in_chrg=<optimized out>)
at
/home/even/gdal/MiraMon-Vector-driver/ogr/ogrsf_frmts/miramon/ogrmiramondatasource.cpp:50
#8 0x00007ffff6f2b014 in OGRMiraMonDataSource::~OGRMiraMonDataSource
(this=0x555556265cb0, __in_chrg=<optimized out>)
at
/home/even/gdal/MiraMon-Vector-driver/ogr/ogrsf_frmts/miramon/ogrmiramondatasource.cpp:57
#9 0x00007ffff73d61a3 in GDALClose (hDS=0x555556265cb0) at
/home/even/gdal/MiraMon-Vector-driver/gcore/gdaldataset.cpp:4128
#10 0x000055555555e167 in TestCreateLayer (poDriver=0x55555564e3f0,
eGeomType=wkbMultiPoint) at
/home/even/gdal/MiraMon-Vector-driver/apps/test_ogrsf.cpp:977
#11 0x000055555555ec61 in TestCreate (poDriver=0x55555564e3f0,
bFromAllDrivers=1) at
/home/even/gdal/MiraMon-Vector-driver/apps/test_ogrsf.cpp:1094
#12 0x000055555555bd56 in ThreadFunctionInternal
(psContext=0x7fffffffdb60) at
/home/even/gdal/MiraMon-Vector-driver/apps/test_ogrsf.cpp:325
#13 0x000055555555bc26 in ThreadFunction (user_data=0x7fffffffdb60) at
/home/even/gdal/MiraMon-Vector-driver/apps/test_ogrsf.cpp:283
#14 0x000055555555ba62 in main (nArgc=2, papszArgv=0x5555556521b0) at
/home/even/gdal/MiraMon-Vector-driver/apps/test_ogrsf.cpp:233
Actually looking at your MMMoveFromFileToFile(), I see it is allocating
a working buffer of 100 MB. That's much larger than what you really
need. A buffer of 64 KB or so would be equally performant. This function
might also leak the buffer at line 2705.
"INFO: MiraMonVector: Creation of /foo/test.pol should have failed."
also indicates some logic error in your driver.
and
$ gdb --args ogr2ogr /foo/test.pol autotest/ogr/data/poly.shp
crashes at
ERROR 4: Error pMMPolygonLayer->pF: Cannot open file /foo/test.pol.
Program received signal SIGSEGV, Segmentation fault.
MMAddArcRecordToMMDB (hMiraMonLayer=0x7fffebbe48f0,
hMMFeature=0x7fffebc030d0, nElemCount=0, pArcHeader=0x555555af3940) at
/home/even/gdal/MiraMon-Vector-driver/ogr/ogrsf_frmts/miramon/mm_wrlayr.c:6602
6602 pMMArcLayer->MMAdmDB.pMMBDXP->nRecords + 1))
(gdb) bt
#0 MMAddArcRecordToMMDB (hMiraMonLayer=0x7fffebbe48f0,
hMMFeature=0x7fffebc030d0, nElemCount=0, pArcHeader=0x555555af3940)
at
/home/even/gdal/MiraMon-Vector-driver/ogr/ogrsf_frmts/miramon/mm_wrlayr.c:6602
#1 0x00007ffff6f3ccd6 in MMCreateFeaturePolOrArc
(hMiraMonLayer=0x7fffebbe48f0, hMMFeature=0x7fffebc030d0) at
/home/even/gdal/MiraMon-Vector-driver/ogr/ogrsf_frmts/miramon/mm_wrlayr.c:4112
#2 0x00007ffff6f3de4e in MMAddFeature (hMiraMonLayer=0x7fffebbe48f0,
hMiraMonFeature=0x7fffebc030d0) at
/home/even/gdal/MiraMon-Vector-driver/ogr/ogrsf_frmts/miramon/mm_wrlayr.c:4598
#3 0x00007ffff6f31787 in OGRMiraMonLayer::MMWriteGeometry
(this=0x7fffebbc6010) at
/home/even/gdal/MiraMon-Vector-driver/ogr/ogrsf_frmts/miramon/ogrmiramonlayer.cpp:1718
#4 0x00007ffff6f30e01 in OGRMiraMonLayer::MMProcessGeometry
(this=0x7fffebbc6010, hGeom=0x555555a30e10, poFeature=0x555555a2d7c0,
bcalculateRecord=1 '\001')
at
/home/even/gdal/MiraMon-Vector-driver/ogr/ogrsf_frmts/miramon/ogrmiramonlayer.cpp:1497
#5 0x00007ffff6f30a8c in OGRMiraMonLayer::MMProcessMultiGeometry
(this=0x7fffebbc6010, hGeom=0x555555a30e10, poFeature=0x555555a2d7c0)
at
/home/even/gdal/MiraMon-Vector-driver/ogr/ogrsf_frmts/miramon/ogrmiramonlayer.cpp:1389
#6 0x00007ffff6f31027 in OGRMiraMonLayer::ICreateFeature
(this=0x7fffebbc6010, poFeature=0x555555a2d7c0) at
/home/even/gdal/MiraMon-Vector-driver/ogr/ogrsf_frmts/miramon/ogrmiramonlayer.cpp:1549
#7 0x00007ffff735ec23 in OGRLayer::CreateFeature (this=0x7fffebbc6010,
poFeature=0x555555a2d7c0) at
/home/even/gdal/MiraMon-Vector-driver/ogr/ogrsf_frmts/generic/ogrlayer.cpp:733
#8 0x00007ffff75c45fc in LayerTranslator::Translate
(this=0x7fffffffd9a0, poFeatureIn=0x0, psInfo=0x555555a2b700,
nCountLayerFeatures=0, pnReadFeatureCount=0x0,
nTotalEventsDone=@0x7fffffffd618: 0,
pfnProgress=0x0, pProgressArg=0x0, psOptions=0x555555640dc0) at
/home/even/gdal/MiraMon-Vector-driver/apps/ogr2ogr_lib.cpp:6292
#9 0x00007ffff75b9d7b in GDALVectorTranslate (pszDest=0x555555640348
"/foo/test.pol", hDstDS=0x0, nSrcCount=1, pahSrcDS=0x7fffffffdaf0,
psOptionsIn=0x5555556403c0, pbUsageError=0x7fffffffdadc)
at /home/even/gdal/MiraMon-Vector-driver/apps/ogr2ogr_lib.cpp:3459
#10 0x000055555555a730 in main (nArgc=3, papszArgv=0x5555556401b0) at
/home/even/gdal/MiraMon-Vector-driver/apps/ogr2ogr_bin.cpp:451
By the way, you might want to rebase again against latest master. I've
just merged https://github.com/OSGeo/gdal/pull/9454 which re-activate
the ASAN job that I had disabled yesterday night. We had some flakiness
lately with the ASAN job. hopefully it is more stable now...
Le 14/03/2024 à 09:51, Abel Pau a écrit :
>
> Sorry,
>
> I merged from my phone and I think I did it wrong.
>
> At this time actions are going well except the same that didn’t work
> at the beginning of this message. The MacOS one.
>
> Merge remote-tracking branch 'upstream/master' into MiraMon-Vector-dr…
> · AbelPau/gdal at fa6f9e0 (github.com)
> <https://github.com/AbelPau/gdal/actions/runs/8277097153/job/22646788426#step:6:2463>
>
> Your proposal of rebasing didn’t solved the problem in macOS.
>
> So, I’ll try to get an image of a mac and compile, but when I tried to
> use a Virtual Box it was extremly slow.
>
> Thanks for any input to try to get some call stack with the error
> described below.
>
> Using gdb I obtain
>
> ogr/ogr_basic_test.py ✓100% ██████████
>
> Results (7.72s):
>
> 1 passed
>
> [Inferior 1 (process 15792) exited normally]
>
> And the same with Valgrind.
>
> Thanks!
>
> ______________________________ test_ogr_basic_10
> _______________________________
>
> 2464
> <https://github.com/AbelPau/gdal/actions/runs/8277097153/job/22646788426#step:6:2465>42:
>
>
> 2465
> <https://github.com/AbelPau/gdal/actions/runs/8277097153/job/22646788426#step:6:2466>42:
> def test_ogr_basic_10():
>
> 2466
> <https://github.com/AbelPau/gdal/actions/runs/8277097153/job/22646788426#step:6:2467>42:
>
>
> 2467
> <https://github.com/AbelPau/gdal/actions/runs/8277097153/job/22646788426#step:6:2468>42:
> import test_cli_utilities
>
> 2468
> <https://github.com/AbelPau/gdal/actions/runs/8277097153/job/22646788426#step:6:2469>42:
>
>
> 2469
> <https://github.com/AbelPau/gdal/actions/runs/8277097153/job/22646788426#step:6:2470>42:
> if test_cli_utilities.get_test_ogrsf_path() is None:
>
> 2470
> <https://github.com/AbelPau/gdal/actions/runs/8277097153/job/22646788426#step:6:2471>42:
> pytest.skip()
>
> 2471
> <https://github.com/AbelPau/gdal/actions/runs/8277097153/job/22646788426#step:6:2472>42:
>
>
> 2472
> <https://github.com/AbelPau/gdal/actions/runs/8277097153/job/22646788426#step:6:2473>42:
> ret = gdaltest.runexternal(
>
> 2473
> <https://github.com/AbelPau/gdal/actions/runs/8277097153/job/22646788426#step:6:2474>42:
> test_cli_utilities.get_test_ogrsf_path() + " -all_drivers"
>
> 2474
> <https://github.com/AbelPau/gdal/actions/runs/8277097153/job/22646788426#step:6:2475>42:
> )
>
> 2475
> <https://github.com/AbelPau/gdal/actions/runs/8277097153/job/22646788426#step:6:2476>42:
>
>
> 2476
> <https://github.com/AbelPau/gdal/actions/runs/8277097153/job/22646788426#step:6:2477>42:
> > assert "INFO" in ret
>
> 2477
> <https://github.com/AbelPau/gdal/actions/runs/8277097153/job/22646788426#step:6:2478>42:
> E AssertionError: assert 'INFO' in '\nERROR ret code = -4'
>
> 2478
> <https://github.com/AbelPau/gdal/actions/runs/8277097153/job/22646788426#step:6:2479>42:
>
>
> 2479
> <https://github.com/AbelPau/gdal/actions/runs/8277097153/job/22646788426#step:6:2480>42:
> /Users/runner/work/gdal/gdal/build/autotest/ogr/ogr_basic_test.py:454:
> AssertionError
>
> *De:*gdal-dev <gdal-dev-bounces at lists.osgeo.org> *En nombre de *Abel
> Pau via gdal-dev
> *Enviado el:* dijous, 14 de març de 2024 7:44
> *Para:* Even Rouault <even.rouault at spatialys.com>;
> gdal-dev at lists.osgeo.org
> *Asunto:* Re: [gdal-dev] There's something strange about two tests
> that are very similar
>
> Hi,
>
> I rebased yesterday, after your message, but same error appeared and
> some more.
>
> I rebased al 7.00 again and it seems that there are again some errors
> in linux builds.
>
> I assume it's because of lasts code updates. I'll wait until something
> stable to rebase again.
>
> Enviat des de l'Outlook per a l'Android <https://aka.ms/AAb9ysg>
>
> ------------------------------------------------------------------------
>
> *From:*Even Rouault <even.rouault at spatialys.com
> <mailto:even.rouault at spatialys.com>>
> *Sent:* Thursday, March 14, 2024 12:15:38 AM
> *To:* Abel Pau <a.pau at creaf.uab.cat <mailto:a.pau at creaf.uab.cat>>;
> gdal-dev at lists.osgeo.org
> <mailto:gdal-dev at lists.osgeo.org><gdal-dev at lists.osgeo.org
> <mailto:gdal-dev at lists.osgeo.org>>
> *Subject:* Re: [gdal-dev] There's something strange about two tests
> that are very similar
>
> try rebasing on top of latest master. It looks like the errors are
> only those fixed per
> https://github.com/OSGeo/gdal/commit/6703d3071de7155d320a39a580f27230428dcaca
>
>
> --
> http://www.spatialys.com
> My software is free, but my time generally not.
--
http://www.spatialys.com
My software is free, but my time generally not.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20240314/30309143/attachment-0001.htm>
More information about the gdal-dev
mailing list