<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<p>Hi,</p>
<p>Not that this will give you answers, but I've just tried the
procedure of
<a class="moz-txt-link-freetext" href="https://llvm.org/docs/ScudoHardenedAllocator.html#library">https://llvm.org/docs/ScudoHardenedAllocator.html#library</a> to build
Scudo as a memory allocator on Linux, and I can ran the
autotest/ogr/ogr_s57.py tests or test_ogrsf some.000 under it just
fine.</p>
<p>Did you try to play with the "release_to_os_interval_ms" setting
of Scudo, like lowering it ?<br>
</p>
<p>I'm wondering if using some malloc recording/replay tooling
couldn't be of help to try to isolate memory allocation from the
logic of the code, and possibly use that as a way for Scudo folks
to replicate if that turns to be a Scudo bug. Random googling led
to <a class="moz-txt-link-freetext" href="https://github.com/alk/malloc-trace-replay">https://github.com/alk/malloc-trace-replay</a>. I've never have
used such thing. Might not be applicable</p>
<p>Even</p>
<p><br>
</p>
<div class="moz-cite-prefix">Le 02/04/2022 à 10:17, Philippe Lelong
a écrit :<br>
</div>
<blockquote type="cite"
cite="mid:eef2f3a56cd34275bd6366caa63e9a88@meltemus.com">
<div id="divtagdefaultwrapper" dir="ltr">
<p>Hi again,</p>
<p><br>
</p>
<p>Still fighting with that one.</p>
<p><br>
</p>
<p>I have tried to not run gdal multithreaded, same issue. I
have also tried to replace malloc(nSize) with calloc(1, nSize)
in VSIMalloc to make sure the memory is initialized, same
issue. And many other things... still crashing.</p>
<p><br>
</p>
<p>I don't know how to interpret the logs I joined in the
previous mail with VSIMalloc etc debugging, do you see
anything special in them?</p>
<p><br>
</p>
<p>Any other idea I can try to isolate the problem?</p>
<p><br>
</p>
<p>Thanks in advance</p>
<p>Philippe.</p>
<br>
<br>
<div>
<hr tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><b>From:</b> gdal-dev
<a class="moz-txt-link-rfc2396E" href="mailto:gdal-dev-bounces@lists.osgeo.org"><gdal-dev-bounces@lists.osgeo.org></a> on behalf of
Philippe Lelong <a class="moz-txt-link-rfc2396E" href="mailto:lelong.ph@meltemus.com"><lelong.ph@meltemus.com></a><br>
<b>Sent:</b> Tuesday, March 29, 2022 7:15 AM<br>
<b>To:</b> <a class="moz-txt-link-abbreviated" href="mailto:gdal-dev@lists.osgeo.org">gdal-dev@lists.osgeo.org</a><br>
<b>Subject:</b> Re: [gdal-dev] Memory allocation issues on
Android 11+ and scudo
<div> </div>
</div>
<div>
<div id="divtagdefaultwrapper" dir="ltr">
<p>Hi,</p>
<p><span>I have some results.</span><br>
</p>
<p><br>
</p>
<p>First of all note that fprintf(stderr,...) does
nothing on Android since it is redirected to /dev/null.
I had to modify cpl_simple.txt in order to use </p>
<p>
__android_log_print(ANDROID_LOG_INFO,LOG_TAG, ...
instead.<br>
</p>
<p>
<br>
</p>
<p>
You can find the full logs here: <a
href="https://www.virtual-winds.org/maitai/gdal_crash.zip"
class="OWAAutoLink moz-txt-link-freetext"
id="LPlnk751975" moz-do-not-send="true">https://www.virtual-winds.org/maitai/gdal_crash.zip</a></p>
<p>
<br>
</p>
<p>
Here is the end of one of them, just before the crash:</p>
<p>
<br>
</p>
<p>
<br>
</p>
<div>
<div>03-28 21:02:14.430 20211 20268 E GDAL_VSI:
Thread[0x7765183cb0] VSIRealloc(0x7736492fd0, 14768) =
0x7736489630, current_cumul = 22725820, mal+cal-free =
163282</div>
<div>03-28 21:02:14.430 20211 20268 E GDAL_VSI:
Thread[0x7765183cb0] VSIRealloc(0x7736489630, 14784) =
0x7736489630, current_cumul = 22725836, mal+cal-free =
163282</div>
<div>03-28 21:02:14.430 20211 20268 E GDAL_VSI:
Thread[0x7765183cb0] VSIMalloc(14784) = 0x773645f620,
current_cumul = 22740668, mal+cal-free = 163283</div>
<div>03-28 21:02:14.430 20211 20268 E GDAL_VSI:
Thread[0x7765183cb0] VSIRealloc(0x773645f620, 14800) =
0x773645f620, current_cumul = 22740684, mal+cal-free =
163284</div>
<div>03-28 21:02:14.430 20211 20268 E GDAL_VSI:
Thread[0x7765183cb0] VSIRealloc(0x773645f620, 29568) =
0x7736455c60, current_cumul = 22755452, mal+cal-free =
163284</div>
<div>03-28 21:02:14.430 20211 20268 E GDAL_VSI:
Thread[0x7765183cb0] VSIRealloc(0x7736455c60, 29584) =
0x7736455c60, current_cumul = 22755468, mal+cal-free =
163284</div>
<div>03-28 21:02:14.430 20211 20268 E GDAL_VSI:
Thread[0x7765183cb0] VSIFree(0x773669f630, (14784
bytes))</div>
<div>03-28 21:02:14.430 20211 20268 E GDAL_VSI:
Thread[0x7765183cb0] VSIFree(0x7736489630, (14784
bytes))</div>
<div>03-28 21:02:14.431 1781 5039 D BtGatt.ContextMap:
remove() - id: 10</div>
<div>03-28 21:02:14.431 1781 5039 E BtGatt.ContextMap:
remove() - removed: 10</div>
<div>03-28 21:02:14.432 1323 1567 D ActivityManager:
Received SERVICE intent 0xa7cee6b Key{startService
pkg=com.sec.android.app.shealth
intent=act=com.samsung.android.app.shealth.tracker.pedometer.PedometerService.HBA
cmp=com.sec.android.app.shealth/com.samsung.android.app.shealth.tracker.pedometer.service.PedometerService
flags=0x0 u=0} requestCode=0 from uid 1000</div>
<div>03-28 21:02:14.433 1781 3013 I bt_stack:
[<a class="moz-txt-link-freetext" href="INFO:gatt_api.cc(1102)">INFO:gatt_api.cc(1102)</a>] GATT_Deregister gatt_if=10</div>
<div>03-28 21:02:14.434 3224 3224 W
[MCFServer]_BleAdapterCallbackManager: onCallbackDied
- client was dead, unregister bleAdapter callback
appId:17</div>
<div>03-28 21:02:14.435 3224 3224 I
[MCFServer]_McfMainController: destroyBleAdapter -
appId=17 , mBleAdapterCallbacks size:0</div>
<div>03-28 21:02:14.435 3224 3224 I
[MCFServer]_McfMainController: destroyBleAdapter -
mBleAdapterCallbacks is empty,
destroyBleAdapterManager after 3s</div>
<div>03-28 21:02:14.435 1781 3013 I bt_stack:
[<a class="moz-txt-link-freetext" href="INFO:gatt_api.cc(1155)">INFO:gatt_api.cc(1155)</a>] Initialize tGATT_REG</div>
<div>03-28 21:02:14.438 1781 2610 E
BtGatt.GattService: [GSIM LOG]: gsimLogHandler, msg:
MESSAGE_SCAN_STOP, appName:
com.samsung.android.mcfserver, scannerId: 10,
reportDelayMillis=0</div>
<div>03-28 21:02:14.440 1323 1567 V
SamsungAlarmManager: setLocked to kernel - W:94823351
/ NW:94183790, now=94162423</div>
<div>03-28 21:02:14.441 1323 1323 W Looper : Drained</div>
<div>03-28 21:02:14.441 1323 3033 W
LocationManagerService: onFreezeStateChanged,
uid[10400]=false</div>
<div>03-28 21:02:14.441 1323 3033 I
PowerManagerService: [PWL] SetWakeLockEnableDisable
uid = 10400 , disable= false</div>
<div>03-28 21:02:14.441 1323 3033 I
PowerManagerService: [PWL] can not change uid = 10400</div>
<div>03-28 21:02:14.442 1323 1360 V
SamsungAlarmManager: setLocked to kernel - W:94823351
/ NW:94183790, now=94162425</div>
<div>03-28 21:02:14.444 1323 1546 W system_server:
Long monitor contention with owner AlarmManager (1567)
at void
com.android.server.alarm.AlarmManagerService$AlarmThread.run()(AlarmManagerService.java:5025)
waiters=0 in void
com.android.server.alarm.AlarmManagerService$DeliveryTracker.alarmComplete(android.os.IBinder)
for 296ms</div>
<div>03-28 21:02:14.444 857 857 D Zygote : Forked
child process 21000</div>
<div>03-28 21:02:14.444 1323 3033 W
LocationManagerService: onFreezeStateChanged,
uid[10301]=false</div>
<div>03-28 21:02:14.445 1323 3033 I
PowerManagerService: [PWL] SetWakeLockEnableDisable
uid = 10301 , disable= false</div>
<div>03-28 21:02:14.445 1323 1432 I ActivityManager:
Start proc 21000:android:drmService/1000 for service
{android/com.android.server.DrmEventService}</div>
<div>03-28 21:02:14.445 1323 3033 I
PowerManagerService: [PWL] can not change uid = 10301</div>
<div>03-28 21:02:14.445 1323 1431 W BroadcastQueue:
Skipping deliver [background] BroadcastRecord{9698798
u-1 android.intent.action.BATTERY_CHANGED} to
ReceiverList{8e48bc4 19599 com.facebook.orca/10400/u0
remote:393b6d7}: process gone or crashing</div>
<div>03-28 21:02:14.445 1323 1431 W BroadcastQueue:
Skipping deliver [background] BroadcastRecord{9698798
u-1 android.intent.action.BATTERY_CHANGED} to
ReceiverList{687dc5e 19599 com.facebook.orca/10400/u0
remote:b49e199}: process gone or crashing</div>
<div>03-28 21:02:14.446 1323 1431 W BroadcastQueue:
Skipping deliver [background] BroadcastRecord{cf3bf1
u-1 android.intent.action.BATTERY_CHANGED} to
ReceiverList{26d4e3d 19145
com.facebook.katana/10301/u0 remote:87e8394}: process
gone or crashing</div>
<div>03-28 21:02:14.446 1323 3033 W
LocationManagerService: onFreezeStateChanged,
uid[10382]=false</div>
<div>03-28 21:02:14.446 1323 3033 I
PowerManagerService: [PWL] SetWakeLockEnableDisable
uid = 10382 , disable= false</div>
<div>03-28 21:02:14.446 1323 3033 I
PowerManagerService: [PWL] can not change uid = 10382</div>
<div>03-28 21:02:14.447 1809 1809 D QS :
setQSExpansion 0.0 -122.85</div>
<div>03-28 21:02:14.449 8549 8549 I PedometerService:
onStartCommand true, true, true, 1648400236406,
1648400236842, false, false</div>
<div>03-28 21:02:14.451 1809 1809 V
SecQSFragmentAnimatorBase: setQsExpansionPosition 0.0</div>
<div>03-28 21:02:14.451 1809 1809 V QsExpandAnimator:
setQsExpansionPosition 0.0 0</div>
<div>03-28 21:02:14.451 1323 1431 W BroadcastQueue:
Skipping deliver [background] BroadcastRecord{cf3bf1
u-1 android.intent.action.BATTERY_CHANGED} to
ReceiverList{86abcc6 19145
com.facebook.katana/10301/u0 remote:8f3e2a1}: process
gone or crashing</div>
<div>03-28 21:02:14.452 1323 1360 D
SamsungAlarmManager: setInexact (T:3/F:0/AC:false)
20220329T000214 now=94162435 -
CU:10251/CP:2699/OP:PendingIntent{9e623d6:
PendingIntentRecord{650d89a
com.google.android.gms/com.google.android.gms.tron
broadcastIntent}}</div>
<div>03-28 21:02:14.452 1323 1360 V
SamsungAlarmManager: setLocked to kernel - W:94823351
/ NW:94183790, now=94162435</div>
<div>03-28 21:02:14.457 1323 2605 D SemContextService:
lock : requestToUpdate</div>
<div>03-28 21:02:14.458 1323 2605 D
SemContext.CaeProvider.SensorStatusCheckImpl: Sensor
Check Event is null!!</div>
<div>03-28 21:02:14.458 1323 2605 D
SemContextService: .requestToUpdate() : service =
Sensor Status Check</div>
<div>03-28 21:02:14.458 1323 2605 D SemContextService:
unlock : requestToUpdate</div>
<div>03-28 21:02:14.464 1809 1809 D QS :
setQSExpansion 0.0 -122.85</div>
<div>03-28 21:02:14.466 8549 31023 I
SHEALTH#WI#WearableConnectionMonitor: (print) :
getConnectedWearableDeviceList(), size : 1</div>
<div>03-28 21:02:14.466 20211 20264 E GDAL_VSI:
Thread[0x776957bcb0] VSIRealloc(0x78ab48f090, 13504) =
0x77341bbb20, current_cumul = 22743760, mal+cal-free =
163385</div>
<div>03-28 21:02:14.466 20211 20264 E GDAL_VSI:
Thread[0x776957bcb0] VSIRealloc(0x77341bbb20, 13520) =
0x77341bbb20, current_cumul = 22743776, mal+cal-free =
163385</div>
<div>03-28 21:02:14.470 21000 21000 E android:drmSer:
Not starting debugger since process cannot load the
jdwp agent.</div>
<div>03-28 21:02:14.471 21000 21000 E USNET : USNET:
appName: android:drmService</div>
<div>03-28 21:02:14.471 21000 21000 D ProcessState:
Binder ioctl to enable oneway spam detection failed:
Invalid argument</div>
<div>03-28 21:02:14.473 996 1051 I heimdall:
insert_task_to_group:64, insert tgid 21000 to group
com.google.process.gapps, ret = 0</div>
<div>03-28 21:02:14.476 21000 21000 D ActivityThread:
setConscryptValidator</div>
<div>03-28 21:02:14.476 21000 21000 D ActivityThread:
setConscryptValidator - put</div>
<div>03-28 21:02:14.479 1323 2605 I ActivityManager:
DSS OFF for android</div>
<div>03-28 21:02:14.484 1323 2605 D ActivityManager:
attachApplicationLocked() app=ProcessRecord{60e4cac
21000:android:drmService/1000}
app.isolatedEntryPoint=null instr2=null</div>
<div>03-28 21:02:14.492 1809 1809 W LooperSlow:
RunCallback: type=3,
action=android.view.ViewRootImpl$TraversalRunnable@d06ee61,
token=null, latencyMillis=918, dur=91ms</div>
<div>03-28 21:02:14.494 1809 1809 D
IndicatorGardenInputProperty: updateRotation() prv:-1
>> new:0</div>
<div>03-28 21:02:14.495 1809 1809 D DeviceState:
getDeviceResolutionPixelSize - currentDensity = 450
deviceDensity = 450 initialDisplaySizeFactor = 1440
currentDisplaySizeFactor = 1080 initialDisplayDensity
= 600 proportionalDensity = 450 proportionalPixel = 72</div>
<div>03-28 21:02:14.495 1809 1809 D DeviceState:
getDeviceResolutionPixelSize - currentDensity = 450
deviceDensity = 450 initialDisplaySizeFactor = 1440
currentDisplaySizeFactor = 1080 initialDisplayDensity
= 600 proportionalDensity = 450 proportionalPixel = 27</div>
<div>03-28 21:02:14.495 1809 1809 D DeviceState:
getDeviceResolutionPixelSize - currentDensity = 450
deviceDensity = 450 initialDisplaySizeFactor = 1440
currentDisplaySizeFactor = 1080 initialDisplayDensity
= 600 proportionalDensity = 450 proportionalPixel = 16</div>
<div>03-28 21:02:14.495 1809 1809 E
IndicatorGardenAlgorithmBasicCutout: NOT MATCH !!!!
resourceHeight:74, cutoutHeight:75
([IndicatorGardenInputProperty]
Rotation(0-0,90-1,180-2,270-3)0, Density:2.8125,
ScreenWidthSize:1080, CoverSidePadding:0,
mIndicatorGardenCenterOffset:12,
mCameraCutoutCropSize:0, mGardenPaddingStart:23,
mIndicatorCornerPadding:72, mCameraSidePadding:27,
mCameraTopMargin:16,
DpCutout:DisplayCutout{insets=Rect(0, 75 - 0, 0)
waterfall=Insets{left=0, top=0, right=0, bottom=0}
boundingRect={Bounds=[Rect(0, 0 - 0, 0), Rect(511, 0 -
569, 75), Rect(0, 0 - 0, 0), Rect(0, 0 - 0, 0)]}
cutoutPathParserInfo={CutoutPathParserInfo{displayWidth=1080
displayHeight=2400 density={2.8125} cutoutSpec={M 0, 0
H -10.4 V 26.66666666666667 H 10.4 V 0 H 0 Z @dp}
rotation={0} scale={1.0}}}}) </div>
<div>03-28 21:02:14.495 1809 1809 E
IndicatorGardenAlgorithmBasicCutout: NOT MATCH !!!!
resourceHeight:74, cutoutHeight:75
([IndicatorGardenInputProperty]
Rotation(0-0,90-1,180-2,270-3)0, Density:2.8125,
ScreenWidthSize:1080, CoverSidePadding:0,
mIndicatorGardenCenterOffset:12,
mCameraCutoutCropSize:0, mGardenPaddingStart:23,
mIndicatorCornerPadding:72, mCameraSidePadding:27,
mCameraTopMargin:16,
DpCutout:DisplayCutout{insets=Rect(0, 75 - 0, 0)
waterfall=Insets{left=0, top=0, right=0, bottom=0}
boundingRect={Bounds=[Rect(0, 0 - 0, 0), Rect(511, 0 -
569, 75), Rect(0, 0 - 0, 0), Rect(0, 0 - 0, 0)]}
cutoutPathParserInfo={CutoutPathParserInfo{displayWidth=1080
displayHeight=2400 density={2.8125} cutoutSpec={M 0, 0
H -10.4 V 26.66666666666667 H 10.4 V 0 H 0 Z @dp}
rotation={0} scale={1.0}}}}) </div>
<div>03-28 21:02:14.495 21000 21000 D ActivityThread:
handleBindApplication()++ app=android:drmService</div>
<div>03-28 21:02:14.496 21000 21000 D
CompatibilityChangeReporter: Compat change id
reported: 171979766; UID 1000; state: ENABLED</div>
<div>03-28 21:02:14.505 1809 1809 D SystemUIService:
SYSUI_RAM_OPTIMIZATION onTrimMemory=15</div>
<div>03-28 21:02:14.506 583 583 I lmkd :
2(delay),0(swap), 0(freelimit) memory pressure events
were skipped after a kill!</div>
<div>03-28 21:02:14.506 1809 1809 D StatusBar:
SYSUI_RAM_OPTIMIZATION onTrimMemory=15</div>
<div>03-28 21:02:14.506 1809 1809 D SystemUIService:
onTrimMemory : 15</div>
<div>03-28 21:02:14.506 583 583 I lmkd : cached
0, sandbox(not0) 0</div>
<div>03-28 21:02:14.506 1809 1809 D SystemUIService:
Last Info is 03-28 21:00:08.559. It still remains
until reset time. So skip this.</div>
<div>03-28 21:02:14.506 1809 1809 D SystemUIService:
SYSUI_RAM_OPTIMIZATION onTrimMemory=15</div>
<div>03-28 21:02:14.506 1809 1809 D StatusBar:
SYSUI_RAM_OPTIMIZATION onTrimMemory=15</div>
<div>03-28 21:02:14.506 1809 1809 D SystemUIService:
onTrimMemory : 15</div>
<div>03-28 21:02:14.507 1809 1809 D SystemUIService:
Last Info is 03-28 21:00:08.559. It still remains
until reset time. So skip this.</div>
<div>03-28 21:02:14.507 1809 1809 W Looper : Slow
dispatch took 106ms main
h=android.view.Choreographer$FrameHandler
c=android.view.Choreographer$FrameDisplayEventReceiver@15dac12
m=0</div>
<div>03-28 21:02:14.510 583 583 E libprocessgroup:
set_timerslack_ns write failed: No such process</div>
<div>03-28 21:02:14.511 583 583 I lmkd : Reclaim
'android.process.acore' (27861), uid 10071,
oom_score_adj 850, state 99 to free 33916kB rss,
56816kB swap; reason: low watermark is breached and
swap is low (1848484kB < 838860kB)</div>
<div>03-28 21:02:14.521 20211 20264 E GDAL_VSI:
Thread[0x776957bcb0] VSIFree(0x77341bbb20, (13520
bytes))</div>
<div>03-28 21:02:14.533 857 857 I Zygote : Process
27861 exited due to signal 9 (Killed)</div>
<div>03-28 21:02:14.533 1323 5047 D
InputMethodManagerService: removeClient</div>
<div>03-28 21:02:14.533 583 583 I lmkd : cached
0, sandbox(not0) 0</div>
<div>03-28 21:02:14.535 583 583 E libprocessgroup:
set_timerslack_ns write failed: No such process</div>
<div>03-28 21:02:14.537 1323 2310 I ActivityManager:
Process android.process.acore (pid 27861) has died:
picked CEM (290,316)</div>
<div>03-28 21:02:14.537 1323 1433 I libprocessgroup:
Successfully killed process cgroup uid 10071 pid 27861
in 0ms</div>
<div>03-28 21:02:14.538 583 583 I lmkd : Reclaim
'com.samsung.cmh:CMH' (28089), uid 5004, oom_score_adj
850, state 99 to free 29444kB rss, 53292kB swap;
reason: low watermark is breached and swap is low
(1863132kB < 838860kB)</div>
<div>03-28 21:02:14.539 1781 1781 D HidDeviceService:
handleMessage(): msg.what=8</div>
<div>03-28 21:02:14.540 1809 1809 W LooperSlow:
RunCallback: type=3,
action=android.view.ViewRootImpl$TraversalRunnable@fe916b9,
token=null, latencyMillis=31, dur=31ms</div>
<div>03-28 21:02:14.540 1809 1809 W Looper : Slow
dispatch took 31ms main
h=android.view.Choreographer$FrameHandler
c=android.view.Choreographer$FrameDisplayEventReceiver@15dac12
m=0</div>
<div>03-28 21:02:14.541 1809 1809 W Looper : Drained</div>
<div>03-28 21:02:14.557 20211 20264 E GDAL_VSI:
Thread[0x776957bcb0] VSIRealloc(0x789d45b450, 21280) =
0x772f097cc0, current_cumul = 22758320, mal+cal-free =
163504</div>
<div>03-28 21:02:14.557 20211 20264 E GDAL_VSI:
Thread[0x776957bcb0] VSIRealloc(0x772f097cc0, 21296) =
0x772f097cc0, current_cumul = 22758336, mal+cal-free =
163504</div>
<div>03-28 21:02:14.558 20211 20269 E GDAL_VSI:
Thread[0x7764085cb0] VSIFree(0x7a89741610, (55312
bytes))</div>
<div>03-28 21:02:14.566 857 857 I Zygote : Process
28089 exited due to signal 9 (Killed)</div>
<div>03-28 21:02:14.566 1323 1553 I ActivityManager:
Process com.samsung.cmh:CMH (pid 28089) has died:
picked CEM (277,320)</div>
<div>03-28 21:02:14.566 583 583 I lmkd : cached
0, sandbox(not0) 0</div>
<div>03-28 21:02:14.566 20211 20262 E GDAL_VSI:
Thread[0x7770e3dcb0] VSIRealloc(0x7a3f23d980, 57120) =
0x7a3f23d980, current_cumul = 22716708, mal+cal-free =
163501</div>
<div>03-28 21:02:14.566 1323 1433 I libprocessgroup:
Successfully killed process cgroup uid 5004 pid 28089
in 0ms</div>
<div>03-28 21:02:14.568 1781 1781 D HidDeviceService:
handleMessage(): msg.what=8</div>
<div>03-28 21:02:14.575 583 583 I lmkd : Reclaim
'com.google.android.gms' (18299), uid 10251,
oom_score_adj 800, state 10 to free 41892kB rss,
50312kB swap; reason: low watermark is breached and
swap is low (1871592kB < 838860kB)</div>
<div>03-28 21:02:14.588 20983 20983 V
GraphicsEnvironment: ANGLE Developer option for
'com.google.android.apps.turbo' set to: 'default'</div>
<div>03-28 21:02:14.594 1809 3054 I OpenGLRenderer:
Davey! duration=989ms; Flags=0,
FrameTimelineVsyncId=1355581,
IntendedVsync=50486826840932, Vsync=50487676840898,
InputEventId=0, HandleInputStart=50487682229734,
AnimationStart=50487682230811,
PerformTraversalsStart=50487682231618,
DrawStart=50487777157695,
FrameDeadline=50486843507598,
FrameInterval=50487682147349, FrameStartTime=16666666,
SyncQueued=50487779379772, SyncStart=50487780687541,
IssueDrawCommandsStart=50487786418772,
SwapBuffers=50487808932657,
FrameCompleted=50487817955695,
DequeueBufferDuration=1428731,
QueueBufferDuration=4194269,
GpuCompleted=50487811641964,
SwapBuffersCompleted=50487817955695,
DisplayPresentTime=0, </div>
<div>03-28 21:02:14.608 1323 2605 D
ConnectivityService: ConnectivityService
NetworkRequestInfo binderDied(uid/pid:10251/18299,
[NetworkRequest [ LISTEN id=3417, [ Transports: WIFI
Capabilities:
NOT_RESTRICTED&TRUSTED&NOT_VPN&NOT_VCN_MANAGED
Uid: 10251 RequestorUid: 10251 RequestorPkg:
com.google.android.gms UnderlyingNetworks: Null] ]],
android.os.BinderProxy@f2b50f3)</div>
<div>03-28 21:02:14.608 583 583 I lmkd : cached
0, sandbox(not0) 0</div>
<div>03-28 21:02:14.609 857 857 I Zygote : Process
18299 exited due to signal 9 (Killed)</div>
<div>03-28 21:02:14.610 1323 2457 D
ConnectivityService: ConnectivityService
NetworkRequestInfo binderDied(uid/pid:10251/18299,
[NetworkRequest [ LISTEN id=3418, [ Transports: WIFI
Capabilities:
NOT_RESTRICTED&TRUSTED&NOT_VPN&NOT_VCN_MANAGED
Uid: 10251 RequestorUid: 10251 RequestorPkg:
com.google.android.gms UnderlyingNetworks: Null] ]],
android.os.BinderProxy@70db1b0)</div>
<div>03-28 21:02:14.610 1323 2605 I ActivityManager:
Process com.google.android.gms (pid 18299) has died:
svc SVC (283,318)</div>
<div>03-28 21:02:14.611 1323 1433 I libprocessgroup:
Successfully killed process cgroup uid 10251 pid 18299
in 0ms</div>
<div>03-28 21:02:14.611 1323 2605 W ActivityManager:
Scheduling restart of crashed service
com.google.android.gms/.cast.service.CastPersistentService
in 50492ms for start-requested</div>
<div>03-28 21:02:14.613 1323 5045 E
WifiMulticastLockManager: Multicaster binderDied</div>
<div>03-28 21:02:14.615 583 583 E libprocessgroup:
set_timerslack_ns write failed: No such process</div>
<div>03-28 21:02:14.616 20211 20267 W libc :
malloc(40) failed: returning null pointer</div>
<div>03-28 21:02:14.616 20211 20263 W libc :
malloc(40) failed: returning null pointer</div>
<div>03-28 21:02:14.616 20211 20266 W libc :
malloc(34) failed: returning null pointer</div>
<div>03-28 21:02:14.616 20211 20269 W libc :
malloc(72) failed: returning null pointer</div>
<div>03-28 21:02:14.616 20211 20268 W libc :
malloc(32) failed: returning null pointer</div>
<div>03-28 21:02:14.616 20211 20262 W libc :
malloc(68) failed: returning null pointer</div>
<div>03-28 21:02:14.616 20211 20264 W libc :
malloc(32) failed: returning null pointer</div>
<div>03-28 21:02:14.617 20211 20265 W libc :
malloc(32) failed: returning null pointer</div>
<div>03-28 21:02:14.617 20211 20269 W libc :
malloc(128) failed: returning null pointer</div>
<div>--------- beginning of crash</div>
<div>03-28 21:02:14.617 20211 20262 F libc : Fatal
signal 6 (SIGABRT), code -1 (SI_QUEUE) in tid 20262
(Thread (pooled)), pid 20211 (.meltemus.qtvlm)</div>
<div>03-28 21:02:14.617 20211 20264 W libc :
malloc(128) failed: returning null pointer</div>
<div><br>
</div>
I hope you can understand something, I don't.</div>
<div><br>
</div>
<div>Best regards and thank</div>
<br>
<br>
<div>
<hr tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><b>From:</b> gdal-dev
<a class="moz-txt-link-rfc2396E" href="mailto:gdal-dev-bounces@lists.osgeo.org"><gdal-dev-bounces@lists.osgeo.org></a> on behalf of
Philippe Lelong <a class="moz-txt-link-rfc2396E" href="mailto:lelong.ph@meltemus.com"><lelong.ph@meltemus.com></a><br>
<b>Sent:</b> Monday, March 28, 2022 3:50 PM<br>
<b>To:</b> <a class="moz-txt-link-abbreviated" href="mailto:gdal-dev@lists.osgeo.org">gdal-dev@lists.osgeo.org</a><br>
<b>Subject:</b> Re: [gdal-dev] Memory allocation
issues on Android 11+ and scudo
<div> </div>
</div>
<div>
<div id="divtagdefaultwrapper" dir="ltr">
<p>Thanks Even and Greg for your fast replies</p>
<p><br>
</p>
<p>The app and GDAL are built with Android NDK 21 (I
have tried NDK 30 and 31) and is targeting SDK 30.
High memory Android device is 10Gb or even 14Gb
with RAM PLUS. I can add that the app itself is
based on Qt 5.15.8 and runs fine on Windows,
Linux, Raspberry 2/3/4 32bits or 64bits, MacOS,
iOS and Android without Scudo. It is heavily
multithreaded including during GDAL calls.</p>
<p><br>
</p>
<p>I can also add that I have tried a 32bits build
(armv7) running on these armV8 devices, same
problem, although the messages in logcat are a bit
different. I can also say that I am 99.9% sure
this is triggered by GDAL, because our same
exactly S52/57 module reading other formats
without GDAL does not crash.</p>
<p><br>
</p>
<p>I have seen these debug defines in
cpl_simple.cpp. I will enable them, rebuild and
report here. I don't have a scudo device apart
from a emulated one but it is x86-64 and it does
not seem to behave exactly the same as the users'
ARM64 bits (Samsung S21). My feeling is that it is
connected to ARM 64 architecture.</p>
<p><br>
</p>
<p>Thanks again, more later hopefully.</p>
<p>Philippe.</p>
<br>
<br>
<div>
<hr tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><b>From:</b>
Even Rouault <a class="moz-txt-link-rfc2396E" href="mailto:even.rouault@spatialys.com"><even.rouault@spatialys.com></a><br>
<b>Sent:</b> Monday, March 28, 2022 3:24 PM<br>
<b>To:</b> Philippe Lelong;
<a class="moz-txt-link-abbreviated" href="mailto:gdal-dev@lists.osgeo.org">gdal-dev@lists.osgeo.org</a><br>
<b>Subject:</b> Re: [gdal-dev] Memory allocation
issues on Android 11+ and scudo
<div> </div>
</div>
<div>
<p>Hi,</p>
<p>didn't hear about Scudo before, but it seems
it is a LLVM side project: <a
class="moz-txt-link-freetext"
href="https://llvm.org/docs/ScudoHardenedAllocator.html"
id="LPlnk657489" moz-do-not-send="true">
https://llvm.org/docs/ScudoHardenedAllocator.html</a></p>
<p>So perhaps you could build and use it on
Linux as shown in <a
class="moz-txt-link-freetext"
href="https://llvm.org/docs/ScudoHardenedAllocator.html#library"
id="LPlnk92418" moz-do-not-send="true">
https://llvm.org/docs/ScudoHardenedAllocator.html#library</a></p>
<p>Besides a potential bug in the allocator, it
might be that the S57 driver has a memory
allocation pattern that doesn't please Scudo.</p>
<p>Assuming that the problematic memory
allocations are done using GDAL's VSIMalloc()
(and not C++ new/delete), then have a look at
the various #define that you can set at the
top of port/cpl_vsisimple.cpp and can be used
to trace memory allocations<br>
</p>
<p>// Uncomment to check consistent usage of
VSIMalloc(), VSIRealloc(),<br>
// VSICalloc(), VSIFree(), VSIStrdup().<br>
// #define DEBUG_VSIMALLOC<br>
<br>
// Uncomment to compute memory usage
statistics.<br>
// DEBUG_VSIMALLOC must also be defined.<br>
// #define DEBUG_VSIMALLOC_STATS<br>
<br>
// Uncomment to print every memory allocation
or deallocation.<br>
// DEBUG_VSIMALLOC must also be defined.<br>
// #define DEBUG_VSIMALLOC_VERBOSE<br>
<br>
// Number of bytes of the malloc/calloc/free
that triggers a debug trace.<br>
// Can be 0 for all allocs.<br>
#define THRESHOLD_PRINT 10000<br>
<br>
Even<br>
<br>
<br>
</p>
<div class="moz-cite-prefix">Le 28/03/2022 à
14:58, Philippe Lelong a écrit :<br>
</div>
<blockquote type="cite">
<div id="divtagdefaultwrapper" dir="ltr">
<p>Hi,</p>
<p><br>
</p>
<p><span>I am searching for this issue for
months now, and cannot find any
solution.</span><br>
</p>
<p><br>
To make a long story short, we are using
GDAL to decode OGR/S57 charts for years
now. We are facing numerous crashes under
Android 11 and up if and only if this
Android 11 implementation is using SCUDO
as a memory allocator (if jemalloc is used
no problems). We face this problem with an
old GDAL2.1.3 version, so we updated to
GDAL 3.4.1 but the issue is the same.</p>
<p><br>
</p>
<p>What I can see is that the memory grows
exponentially until no more memory is
available and crash, even on systems with
huge memory available while an Android
device without SCUDO and very limited
memory (let's say 4Gb) in the same
exact conditions, with the same apk, runs
perfectly. The logcat command show this:</p>
<p> </p>
<div><span>03-28 12:40:34.255 4959 5005 W
libc : malloc(264196) failed:
returning null pointer</span><br>
</div>
<div>03-28 12:40:34.255 4959 5005 W libc
: malloc(264196) failed: returning null
pointer</div>
<div>03-28 12:40:34.256 4959 5005 W libc
: malloc(264196) failed: returning null
pointer</div>
<div>03-28 12:40:34.256 4959 5005 W libc
: malloc(264196) failed: returning null
pointer</div>
<div>03-28 12:40:34.612 630 630 D
io_stats: !@ Write_top(KB):
kworker/u16:1(32583) 8</div>
<div>03-28 12:40:34.820 4959 5041 I scudo
: Scudo ERROR: out of memory trying to
allocate 64 bytes</div>
<div>03-28 12:40:34.820 4959 5042 I scudo
: Scudo ERROR: out of memory trying to
allocate 64 bytes</div>
<div>03-28 12:40:34.820 4959 5033 I scudo
: Scudo ERROR: out of memory trying to
allocate 64 bytes</div>
<div>03-28 12:40:34.820 4959 5031 I scudo
: Scudo ERROR: out of memory trying to
allocate 64 bytes</div>
<div>03-28 12:40:34.820 4959 5038 I scudo
: Scudo ERROR: out of memory trying to
allocate 64 bytes</div>
<div>03-28 12:40:34.820 4959 5040 I scudo
: Scudo ERROR: out of memory trying to
allocate 64 bytes</div>
<div><br>
</div>
<div>and then crash</div>
<div><br>
</div>
<div>Any help on how to debug and eventually
fix this would be highly appreciated.</div>
<div><br>
</div>
<div>Best regards,</div>
<div>Philippe from qtVlm development team.</div>
<div><br>
</div>
</div>
<br>
<fieldset class="moz-mime-attachment-header"></fieldset>
<pre class="moz-quote-pre">_______________________________________________
gdal-dev mailing list
<a class="moz-txt-link-abbreviated moz-txt-link-freetext" href="mailto:gdal-dev@lists.osgeo.org" moz-do-not-send="true">gdal-dev@lists.osgeo.org</a>
<a class="moz-txt-link-freetext" href="https://lists.osgeo.org/mailman/listinfo/gdal-dev" moz-do-not-send="true">https://lists.osgeo.org/mailman/listinfo/gdal-dev</a>
</pre>
</blockquote>
<pre class="moz-signature" cols="72">--
<a class="moz-txt-link-freetext" href="http://www.spatialys.com" moz-do-not-send="true">http://www.spatialys.com</a>
My software is free, but my time generally not.</pre>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<br>
<fieldset class="moz-mime-attachment-header"></fieldset>
<pre class="moz-quote-pre" wrap="">_______________________________________________
gdal-dev mailing list
<a class="moz-txt-link-abbreviated" href="mailto:gdal-dev@lists.osgeo.org">gdal-dev@lists.osgeo.org</a>
<a class="moz-txt-link-freetext" href="https://lists.osgeo.org/mailman/listinfo/gdal-dev">https://lists.osgeo.org/mailman/listinfo/gdal-dev</a>
</pre>
</blockquote>
<pre class="moz-signature" cols="72">--
<a class="moz-txt-link-freetext" href="http://www.spatialys.com">http://www.spatialys.com</a>
My software is free, but my time generally not.</pre>
</body>
</html>