[gdal-dev] Memory allocation issues on Android 11+ and scudo
Even Rouault
even.rouault at spatialys.com
Sat Apr 2 04:26:45 PDT 2022
Hi,
Not that this will give you answers, but I've just tried the procedure
of https://llvm.org/docs/ScudoHardenedAllocator.html#library to build
Scudo as a memory allocator on Linux, and I can ran the
autotest/ogr/ogr_s57.py tests or test_ogrsf some.000 under it just fine.
Did you try to play with the "release_to_os_interval_ms" setting of
Scudo, like lowering it ?
I'm wondering if using some malloc recording/replay tooling couldn't be
of help to try to isolate memory allocation from the logic of the code,
and possibly use that as a way for Scudo folks to replicate if that
turns to be a Scudo bug. Random googling led to
https://github.com/alk/malloc-trace-replay. I've never have used such
thing. Might not be applicable
Even
Le 02/04/2022 à 10:17, Philippe Lelong a écrit :
>
> Hi again,
>
>
> Still fighting with that one.
>
>
> I have tried to not run gdal multithreaded, same issue. I have also
> tried to replace malloc(nSize) with calloc(1, nSize) in VSIMalloc to
> make sure the memory is initialized, same issue. And many other
> things... still crashing.
>
>
> I don't know how to interpret the logs I joined in the previous mail
> with VSIMalloc etc debugging, do you see anything special in them?
>
>
> Any other idea I can try to isolate the problem?
>
>
> Thanks in advance
>
> Philippe.
>
>
>
> ------------------------------------------------------------------------
> *From:* gdal-dev <gdal-dev-bounces at lists.osgeo.org> on behalf of
> Philippe Lelong <lelong.ph at meltemus.com>
> *Sent:* Tuesday, March 29, 2022 7:15 AM
> *To:* gdal-dev at lists.osgeo.org
> *Subject:* Re: [gdal-dev] Memory allocation issues on Android 11+ and
> scudo
>
> Hi,
>
> I have some results.
>
>
> First of all note that fprintf(stderr,...) does nothing on Android
> since it is redirected to /dev/null. I had to modify cpl_simple.txt in
> order to use
>
> __android_log_print(ANDROID_LOG_INFO,LOG_TAG, ... instead.
>
>
> You can find the full logs here:
> https://www.virtual-winds.org/maitai/gdal_crash.zip
> <https://www.virtual-winds.org/maitai/gdal_crash.zip>
>
>
> Here is the end of one of them, just before the crash:
>
>
>
> 03-28 21:02:14.430 20211 20268 E GDAL_VSI: Thread[0x7765183cb0]
> VSIRealloc(0x7736492fd0, 14768) = 0x7736489630, current_cumul =
> 22725820, mal+cal-free = 163282
> 03-28 21:02:14.430 20211 20268 E GDAL_VSI: Thread[0x7765183cb0]
> VSIRealloc(0x7736489630, 14784) = 0x7736489630, current_cumul =
> 22725836, mal+cal-free = 163282
> 03-28 21:02:14.430 20211 20268 E GDAL_VSI: Thread[0x7765183cb0]
> VSIMalloc(14784) = 0x773645f620, current_cumul = 22740668,
> mal+cal-free = 163283
> 03-28 21:02:14.430 20211 20268 E GDAL_VSI: Thread[0x7765183cb0]
> VSIRealloc(0x773645f620, 14800) = 0x773645f620, current_cumul =
> 22740684, mal+cal-free = 163284
> 03-28 21:02:14.430 20211 20268 E GDAL_VSI: Thread[0x7765183cb0]
> VSIRealloc(0x773645f620, 29568) = 0x7736455c60, current_cumul =
> 22755452, mal+cal-free = 163284
> 03-28 21:02:14.430 20211 20268 E GDAL_VSI: Thread[0x7765183cb0]
> VSIRealloc(0x7736455c60, 29584) = 0x7736455c60, current_cumul =
> 22755468, mal+cal-free = 163284
> 03-28 21:02:14.430 20211 20268 E GDAL_VSI: Thread[0x7765183cb0]
> VSIFree(0x773669f630, (14784 bytes))
> 03-28 21:02:14.430 20211 20268 E GDAL_VSI: Thread[0x7765183cb0]
> VSIFree(0x7736489630, (14784 bytes))
> 03-28 21:02:14.431 1781 5039 D BtGatt.ContextMap: remove() - id: 10
> 03-28 21:02:14.431 1781 5039 E BtGatt.ContextMap: remove() - removed: 10
> 03-28 21:02:14.432 1323 1567 D ActivityManager: Received SERVICE
> intent 0xa7cee6b Key{startService pkg=com.sec.android.app.shealth
> intent=act=com.samsung.android.app.shealth.tracker.pedometer.PedometerService.HBA
> cmp=com.sec.android.app.shealth/com.samsung.android.app.shealth.tracker.pedometer.service.PedometerService
> flags=0x0 u=0} requestCode=0 from uid 1000
> 03-28 21:02:14.433 1781 3013 I bt_stack: [INFO:gatt_api.cc(1102)]
> GATT_Deregister gatt_if=10
> 03-28 21:02:14.434 3224 3224 W
> [MCFServer]_BleAdapterCallbackManager: onCallbackDied - client was
> dead, unregister bleAdapter callback appId:17
> 03-28 21:02:14.435 3224 3224 I [MCFServer]_McfMainController:
> destroyBleAdapter - appId=17 , mBleAdapterCallbacks size:0
> 03-28 21:02:14.435 3224 3224 I [MCFServer]_McfMainController:
> destroyBleAdapter - mBleAdapterCallbacks is empty,
> destroyBleAdapterManager after 3s
> 03-28 21:02:14.435 1781 3013 I bt_stack: [INFO:gatt_api.cc(1155)]
> Initialize tGATT_REG
> 03-28 21:02:14.438 1781 2610 E BtGatt.GattService: [GSIM LOG]:
> gsimLogHandler, msg: MESSAGE_SCAN_STOP, appName:
> com.samsung.android.mcfserver, scannerId: 10, reportDelayMillis=0
> 03-28 21:02:14.440 1323 1567 V SamsungAlarmManager: setLocked to
> kernel - W:94823351 / NW:94183790, now=94162423
> 03-28 21:02:14.441 1323 1323 W Looper : Drained
> 03-28 21:02:14.441 1323 3033 W LocationManagerService:
> onFreezeStateChanged, uid[10400]=false
> 03-28 21:02:14.441 1323 3033 I PowerManagerService: [PWL]
> SetWakeLockEnableDisable uid = 10400 , disable= false
> 03-28 21:02:14.441 1323 3033 I PowerManagerService: [PWL] can not
> change uid = 10400
> 03-28 21:02:14.442 1323 1360 V SamsungAlarmManager: setLocked to
> kernel - W:94823351 / NW:94183790, now=94162425
> 03-28 21:02:14.444 1323 1546 W system_server: Long monitor
> contention with owner AlarmManager (1567) at void
> com.android.server.alarm.AlarmManagerService$AlarmThread.run()(AlarmManagerService.java:5025)
> waiters=0 in void
> com.android.server.alarm.AlarmManagerService$DeliveryTracker.alarmComplete(android.os.IBinder)
> for 296ms
> 03-28 21:02:14.444 857 857 D Zygote : Forked child process 21000
> 03-28 21:02:14.444 1323 3033 W LocationManagerService:
> onFreezeStateChanged, uid[10301]=false
> 03-28 21:02:14.445 1323 3033 I PowerManagerService: [PWL]
> SetWakeLockEnableDisable uid = 10301 , disable= false
> 03-28 21:02:14.445 1323 1432 I ActivityManager: Start proc
> 21000:android:drmService/1000 for service
> {android/com.android.server.DrmEventService}
> 03-28 21:02:14.445 1323 3033 I PowerManagerService: [PWL] can not
> change uid = 10301
> 03-28 21:02:14.445 1323 1431 W BroadcastQueue: Skipping deliver
> [background] BroadcastRecord{9698798 u-1
> android.intent.action.BATTERY_CHANGED} to ReceiverList{8e48bc4 19599
> com.facebook.orca/10400/u0 remote:393b6d7}: process gone or crashing
> 03-28 21:02:14.445 1323 1431 W BroadcastQueue: Skipping deliver
> [background] BroadcastRecord{9698798 u-1
> android.intent.action.BATTERY_CHANGED} to ReceiverList{687dc5e 19599
> com.facebook.orca/10400/u0 remote:b49e199}: process gone or crashing
> 03-28 21:02:14.446 1323 1431 W BroadcastQueue: Skipping deliver
> [background] BroadcastRecord{cf3bf1 u-1
> android.intent.action.BATTERY_CHANGED} to ReceiverList{26d4e3d 19145
> com.facebook.katana/10301/u0 remote:87e8394}: process gone or crashing
> 03-28 21:02:14.446 1323 3033 W LocationManagerService:
> onFreezeStateChanged, uid[10382]=false
> 03-28 21:02:14.446 1323 3033 I PowerManagerService: [PWL]
> SetWakeLockEnableDisable uid = 10382 , disable= false
> 03-28 21:02:14.446 1323 3033 I PowerManagerService: [PWL] can not
> change uid = 10382
> 03-28 21:02:14.447 1809 1809 D QS : setQSExpansion 0.0 -122.85
> 03-28 21:02:14.449 8549 8549 I PedometerService: onStartCommand
> true, true, true, 1648400236406, 1648400236842, false, false
> 03-28 21:02:14.451 1809 1809 V SecQSFragmentAnimatorBase:
> setQsExpansionPosition 0.0
> 03-28 21:02:14.451 1809 1809 V QsExpandAnimator:
> setQsExpansionPosition 0.0 0
> 03-28 21:02:14.451 1323 1431 W BroadcastQueue: Skipping deliver
> [background] BroadcastRecord{cf3bf1 u-1
> android.intent.action.BATTERY_CHANGED} to ReceiverList{86abcc6 19145
> com.facebook.katana/10301/u0 remote:8f3e2a1}: process gone or crashing
> 03-28 21:02:14.452 1323 1360 D SamsungAlarmManager: setInexact
> (T:3/F:0/AC:false) 20220329T000214 now=94162435 -
> CU:10251/CP:2699/OP:PendingIntent{9e623d6: PendingIntentRecord{650d89a
> com.google.android.gms/com.google.android.gms.tron broadcastIntent}}
> 03-28 21:02:14.452 1323 1360 V SamsungAlarmManager: setLocked to
> kernel - W:94823351 / NW:94183790, now=94162435
> 03-28 21:02:14.457 1323 2605 D SemContextService: lock : requestToUpdate
> 03-28 21:02:14.458 1323 2605 D
> SemContext.CaeProvider.SensorStatusCheckImpl: Sensor Check Event is null!!
> 03-28 21:02:14.458 1323 2605 D SemContextService:
> .requestToUpdate() : service = Sensor Status Check
> 03-28 21:02:14.458 1323 2605 D SemContextService: unlock :
> requestToUpdate
> 03-28 21:02:14.464 1809 1809 D QS : setQSExpansion 0.0 -122.85
> 03-28 21:02:14.466 8549 31023 I SHEALTH#WI#WearableConnectionMonitor:
> (print) : getConnectedWearableDeviceList(), size : 1
> 03-28 21:02:14.466 20211 20264 E GDAL_VSI: Thread[0x776957bcb0]
> VSIRealloc(0x78ab48f090, 13504) = 0x77341bbb20, current_cumul =
> 22743760, mal+cal-free = 163385
> 03-28 21:02:14.466 20211 20264 E GDAL_VSI: Thread[0x776957bcb0]
> VSIRealloc(0x77341bbb20, 13520) = 0x77341bbb20, current_cumul =
> 22743776, mal+cal-free = 163385
> 03-28 21:02:14.470 21000 21000 E android:drmSer: Not starting debugger
> since process cannot load the jdwp agent.
> 03-28 21:02:14.471 21000 21000 E USNET : USNET: appName:
> android:drmService
> 03-28 21:02:14.471 21000 21000 D ProcessState: Binder ioctl to enable
> oneway spam detection failed: Invalid argument
> 03-28 21:02:14.473 996 1051 I heimdall: insert_task_to_group:64,
> insert tgid 21000 to group com.google.process.gapps, ret = 0
> 03-28 21:02:14.476 21000 21000 D ActivityThread: setConscryptValidator
> 03-28 21:02:14.476 21000 21000 D ActivityThread: setConscryptValidator
> - put
> 03-28 21:02:14.479 1323 2605 I ActivityManager: DSS OFF for android
> 03-28 21:02:14.484 1323 2605 D ActivityManager:
> attachApplicationLocked() app=ProcessRecord{60e4cac
> 21000:android:drmService/1000} app.isolatedEntryPoint=null instr2=null
> 03-28 21:02:14.492 1809 1809 W LooperSlow: RunCallback: type=3,
> action=android.view.ViewRootImpl$TraversalRunnable at d06ee61,
> token=null, latencyMillis=918, dur=91ms
> 03-28 21:02:14.494 1809 1809 D IndicatorGardenInputProperty:
> updateRotation() prv:-1 >> new:0
> 03-28 21:02:14.495 1809 1809 D DeviceState:
> getDeviceResolutionPixelSize - currentDensity = 450 deviceDensity =
> 450 initialDisplaySizeFactor = 1440 currentDisplaySizeFactor = 1080
> initialDisplayDensity = 600 proportionalDensity = 450
> proportionalPixel = 72
> 03-28 21:02:14.495 1809 1809 D DeviceState:
> getDeviceResolutionPixelSize - currentDensity = 450 deviceDensity =
> 450 initialDisplaySizeFactor = 1440 currentDisplaySizeFactor = 1080
> initialDisplayDensity = 600 proportionalDensity = 450
> proportionalPixel = 27
> 03-28 21:02:14.495 1809 1809 D DeviceState:
> getDeviceResolutionPixelSize - currentDensity = 450 deviceDensity =
> 450 initialDisplaySizeFactor = 1440 currentDisplaySizeFactor = 1080
> initialDisplayDensity = 600 proportionalDensity = 450
> proportionalPixel = 16
> 03-28 21:02:14.495 1809 1809 E IndicatorGardenAlgorithmBasicCutout:
> NOT MATCH !!!! resourceHeight:74, cutoutHeight:75
> ([IndicatorGardenInputProperty] Rotation(0-0,90-1,180-2,270-3)0,
> Density:2.8125, ScreenWidthSize:1080, CoverSidePadding:0,
> mIndicatorGardenCenterOffset:12, mCameraCutoutCropSize:0,
> mGardenPaddingStart:23, mIndicatorCornerPadding:72,
> mCameraSidePadding:27, mCameraTopMargin:16,
> DpCutout:DisplayCutout{insets=Rect(0, 75 - 0, 0)
> waterfall=Insets{left=0, top=0, right=0, bottom=0}
> boundingRect={Bounds=[Rect(0, 0 - 0, 0), Rect(511, 0 - 569, 75),
> Rect(0, 0 - 0, 0), Rect(0, 0 - 0, 0)]}
> cutoutPathParserInfo={CutoutPathParserInfo{displayWidth=1080
> displayHeight=2400 density={2.8125} cutoutSpec={M 0, 0 H -10.4 V
> 26.66666666666667 H 10.4 V 0 H 0 Z @dp} rotation={0} scale={1.0}}}})
> 03-28 21:02:14.495 1809 1809 E IndicatorGardenAlgorithmBasicCutout:
> NOT MATCH !!!! resourceHeight:74, cutoutHeight:75
> ([IndicatorGardenInputProperty] Rotation(0-0,90-1,180-2,270-3)0,
> Density:2.8125, ScreenWidthSize:1080, CoverSidePadding:0,
> mIndicatorGardenCenterOffset:12, mCameraCutoutCropSize:0,
> mGardenPaddingStart:23, mIndicatorCornerPadding:72,
> mCameraSidePadding:27, mCameraTopMargin:16,
> DpCutout:DisplayCutout{insets=Rect(0, 75 - 0, 0)
> waterfall=Insets{left=0, top=0, right=0, bottom=0}
> boundingRect={Bounds=[Rect(0, 0 - 0, 0), Rect(511, 0 - 569, 75),
> Rect(0, 0 - 0, 0), Rect(0, 0 - 0, 0)]}
> cutoutPathParserInfo={CutoutPathParserInfo{displayWidth=1080
> displayHeight=2400 density={2.8125} cutoutSpec={M 0, 0 H -10.4 V
> 26.66666666666667 H 10.4 V 0 H 0 Z @dp} rotation={0} scale={1.0}}}})
> 03-28 21:02:14.495 21000 21000 D ActivityThread:
> handleBindApplication()++ app=android:drmService
> 03-28 21:02:14.496 21000 21000 D CompatibilityChangeReporter: Compat
> change id reported: 171979766; UID 1000; state: ENABLED
> 03-28 21:02:14.505 1809 1809 D SystemUIService:
> SYSUI_RAM_OPTIMIZATION onTrimMemory=15
> 03-28 21:02:14.506 583 583 I lmkd : 2(delay),0(swap),
> 0(freelimit) memory pressure events were skipped after a kill!
> 03-28 21:02:14.506 1809 1809 D StatusBar: SYSUI_RAM_OPTIMIZATION
> onTrimMemory=15
> 03-28 21:02:14.506 1809 1809 D SystemUIService: onTrimMemory : 15
> 03-28 21:02:14.506 583 583 I lmkd : cached 0, sandbox(not0) 0
> 03-28 21:02:14.506 1809 1809 D SystemUIService: Last Info is 03-28
> 21:00:08.559. It still remains until reset time. So skip this.
> 03-28 21:02:14.506 1809 1809 D SystemUIService:
> SYSUI_RAM_OPTIMIZATION onTrimMemory=15
> 03-28 21:02:14.506 1809 1809 D StatusBar: SYSUI_RAM_OPTIMIZATION
> onTrimMemory=15
> 03-28 21:02:14.506 1809 1809 D SystemUIService: onTrimMemory : 15
> 03-28 21:02:14.507 1809 1809 D SystemUIService: Last Info is 03-28
> 21:00:08.559. It still remains until reset time. So skip this.
> 03-28 21:02:14.507 1809 1809 W Looper : Slow dispatch took 106ms
> main h=android.view.Choreographer$FrameHandler
> c=android.view.Choreographer$FrameDisplayEventReceiver at 15dac12 m=0
> 03-28 21:02:14.510 583 583 E libprocessgroup: set_timerslack_ns
> write failed: No such process
> 03-28 21:02:14.511 583 583 I lmkd : Reclaim
> 'android.process.acore' (27861), uid 10071, oom_score_adj 850, state
> 99 to free 33916kB rss, 56816kB swap; reason: low watermark is
> breached and swap is low (1848484kB < 838860kB)
> 03-28 21:02:14.521 20211 20264 E GDAL_VSI: Thread[0x776957bcb0]
> VSIFree(0x77341bbb20, (13520 bytes))
> 03-28 21:02:14.533 857 857 I Zygote : Process 27861 exited due to
> signal 9 (Killed)
> 03-28 21:02:14.533 1323 5047 D InputMethodManagerService: removeClient
> 03-28 21:02:14.533 583 583 I lmkd : cached 0, sandbox(not0) 0
> 03-28 21:02:14.535 583 583 E libprocessgroup: set_timerslack_ns
> write failed: No such process
> 03-28 21:02:14.537 1323 2310 I ActivityManager: Process
> android.process.acore (pid 27861) has died: picked CEM (290,316)
> 03-28 21:02:14.537 1323 1433 I libprocessgroup: Successfully killed
> process cgroup uid 10071 pid 27861 in 0ms
> 03-28 21:02:14.538 583 583 I lmkd : Reclaim
> 'com.samsung.cmh:CMH' (28089), uid 5004, oom_score_adj 850, state 99
> to free 29444kB rss, 53292kB swap; reason: low watermark is breached
> and swap is low (1863132kB < 838860kB)
> 03-28 21:02:14.539 1781 1781 D HidDeviceService: handleMessage():
> msg.what=8
> 03-28 21:02:14.540 1809 1809 W LooperSlow: RunCallback: type=3,
> action=android.view.ViewRootImpl$TraversalRunnable at fe916b9,
> token=null, latencyMillis=31, dur=31ms
> 03-28 21:02:14.540 1809 1809 W Looper : Slow dispatch took 31ms
> main h=android.view.Choreographer$FrameHandler
> c=android.view.Choreographer$FrameDisplayEventReceiver at 15dac12 m=0
> 03-28 21:02:14.541 1809 1809 W Looper : Drained
> 03-28 21:02:14.557 20211 20264 E GDAL_VSI: Thread[0x776957bcb0]
> VSIRealloc(0x789d45b450, 21280) = 0x772f097cc0, current_cumul =
> 22758320, mal+cal-free = 163504
> 03-28 21:02:14.557 20211 20264 E GDAL_VSI: Thread[0x776957bcb0]
> VSIRealloc(0x772f097cc0, 21296) = 0x772f097cc0, current_cumul =
> 22758336, mal+cal-free = 163504
> 03-28 21:02:14.558 20211 20269 E GDAL_VSI: Thread[0x7764085cb0]
> VSIFree(0x7a89741610, (55312 bytes))
> 03-28 21:02:14.566 857 857 I Zygote : Process 28089 exited due to
> signal 9 (Killed)
> 03-28 21:02:14.566 1323 1553 I ActivityManager: Process
> com.samsung.cmh:CMH (pid 28089) has died: picked CEM (277,320)
> 03-28 21:02:14.566 583 583 I lmkd : cached 0, sandbox(not0) 0
> 03-28 21:02:14.566 20211 20262 E GDAL_VSI: Thread[0x7770e3dcb0]
> VSIRealloc(0x7a3f23d980, 57120) = 0x7a3f23d980, current_cumul =
> 22716708, mal+cal-free = 163501
> 03-28 21:02:14.566 1323 1433 I libprocessgroup: Successfully killed
> process cgroup uid 5004 pid 28089 in 0ms
> 03-28 21:02:14.568 1781 1781 D HidDeviceService: handleMessage():
> msg.what=8
> 03-28 21:02:14.575 583 583 I lmkd : Reclaim
> 'com.google.android.gms' (18299), uid 10251, oom_score_adj 800, state
> 10 to free 41892kB rss, 50312kB swap; reason: low watermark is
> breached and swap is low (1871592kB < 838860kB)
> 03-28 21:02:14.588 20983 20983 V GraphicsEnvironment: ANGLE Developer
> option for 'com.google.android.apps.turbo' set to: 'default'
> 03-28 21:02:14.594 1809 3054 I OpenGLRenderer: Davey!
> duration=989ms; Flags=0, FrameTimelineVsyncId=1355581,
> IntendedVsync=50486826840932, Vsync=50487676840898, InputEventId=0,
> HandleInputStart=50487682229734, AnimationStart=50487682230811,
> PerformTraversalsStart=50487682231618, DrawStart=50487777157695,
> FrameDeadline=50486843507598, FrameInterval=50487682147349,
> FrameStartTime=16666666, SyncQueued=50487779379772,
> SyncStart=50487780687541, IssueDrawCommandsStart=50487786418772,
> SwapBuffers=50487808932657, FrameCompleted=50487817955695,
> DequeueBufferDuration=1428731, QueueBufferDuration=4194269,
> GpuCompleted=50487811641964, SwapBuffersCompleted=50487817955695,
> DisplayPresentTime=0,
> 03-28 21:02:14.608 1323 2605 D ConnectivityService:
> ConnectivityService NetworkRequestInfo binderDied(uid/pid:10251/18299,
> [NetworkRequest [ LISTEN id=3417, [ Transports: WIFI Capabilities:
> NOT_RESTRICTED&TRUSTED&NOT_VPN&NOT_VCN_MANAGED Uid: 10251
> RequestorUid: 10251 RequestorPkg: com.google.android.gms
> UnderlyingNetworks: Null] ]], android.os.BinderProxy at f2b50f3)
> 03-28 21:02:14.608 583 583 I lmkd : cached 0, sandbox(not0) 0
> 03-28 21:02:14.609 857 857 I Zygote : Process 18299 exited due to
> signal 9 (Killed)
> 03-28 21:02:14.610 1323 2457 D ConnectivityService:
> ConnectivityService NetworkRequestInfo binderDied(uid/pid:10251/18299,
> [NetworkRequest [ LISTEN id=3418, [ Transports: WIFI Capabilities:
> NOT_RESTRICTED&TRUSTED&NOT_VPN&NOT_VCN_MANAGED Uid: 10251
> RequestorUid: 10251 RequestorPkg: com.google.android.gms
> UnderlyingNetworks: Null] ]], android.os.BinderProxy at 70db1b0)
> 03-28 21:02:14.610 1323 2605 I ActivityManager: Process
> com.google.android.gms (pid 18299) has died: svc SVC (283,318)
> 03-28 21:02:14.611 1323 1433 I libprocessgroup: Successfully killed
> process cgroup uid 10251 pid 18299 in 0ms
> 03-28 21:02:14.611 1323 2605 W ActivityManager: Scheduling restart
> of crashed service
> com.google.android.gms/.cast.service.CastPersistentService in 50492ms
> for start-requested
> 03-28 21:02:14.613 1323 5045 E WifiMulticastLockManager: Multicaster
> binderDied
> 03-28 21:02:14.615 583 583 E libprocessgroup: set_timerslack_ns
> write failed: No such process
> 03-28 21:02:14.616 20211 20267 W libc : malloc(40) failed:
> returning null pointer
> 03-28 21:02:14.616 20211 20263 W libc : malloc(40) failed:
> returning null pointer
> 03-28 21:02:14.616 20211 20266 W libc : malloc(34) failed:
> returning null pointer
> 03-28 21:02:14.616 20211 20269 W libc : malloc(72) failed:
> returning null pointer
> 03-28 21:02:14.616 20211 20268 W libc : malloc(32) failed:
> returning null pointer
> 03-28 21:02:14.616 20211 20262 W libc : malloc(68) failed:
> returning null pointer
> 03-28 21:02:14.616 20211 20264 W libc : malloc(32) failed:
> returning null pointer
> 03-28 21:02:14.617 20211 20265 W libc : malloc(32) failed:
> returning null pointer
> 03-28 21:02:14.617 20211 20269 W libc : malloc(128) failed:
> returning null pointer
> --------- beginning of crash
> 03-28 21:02:14.617 20211 20262 F libc : Fatal signal 6 (SIGABRT),
> code -1 (SI_QUEUE) in tid 20262 (Thread (pooled)), pid 20211
> (.meltemus.qtvlm)
> 03-28 21:02:14.617 20211 20264 W libc : malloc(128) failed:
> returning null pointer
>
> I hope you can understand something, I don't.
>
> Best regards and thank
>
>
> ------------------------------------------------------------------------
> *From:* gdal-dev <gdal-dev-bounces at lists.osgeo.org> on behalf of
> Philippe Lelong <lelong.ph at meltemus.com>
> *Sent:* Monday, March 28, 2022 3:50 PM
> *To:* gdal-dev at lists.osgeo.org
> *Subject:* Re: [gdal-dev] Memory allocation issues on Android 11+ and
> scudo
>
> Thanks Even and Greg for your fast replies
>
>
> The app and GDAL are built with Android NDK 21 (I have tried NDK 30
> and 31) and is targeting SDK 30. High memory Android device is 10Gb or
> even 14Gb with RAM PLUS. I can add that the app itself is based on Qt
> 5.15.8 and runs fine on Windows, Linux, Raspberry 2/3/4 32bits or
> 64bits, MacOS, iOS and Android without Scudo. It is heavily
> multithreaded including during GDAL calls.
>
>
> I can also add that I have tried a 32bits build (armv7) running on
> these armV8 devices, same problem, although the messages in logcat are
> a bit different. I can also say that I am 99.9% sure this is triggered
> by GDAL, because our same exactly S52/57 module reading other formats
> without GDAL does not crash.
>
>
> I have seen these debug defines in cpl_simple.cpp. I will enable them,
> rebuild and report here. I don't have a scudo device apart from a
> emulated one but it is x86-64 and it does not seem to behave
> exactly the same as the users' ARM64 bits (Samsung S21). My feeling is
> that it is connected to ARM 64 architecture.
>
>
> Thanks again, more later hopefully.
>
> Philippe.
>
>
>
> ------------------------------------------------------------------------
> *From:* Even Rouault <even.rouault at spatialys.com>
> *Sent:* Monday, March 28, 2022 3:24 PM
> *To:* Philippe Lelong; gdal-dev at lists.osgeo.org
> *Subject:* Re: [gdal-dev] Memory allocation issues on Android 11+ and
> scudo
>
> Hi,
>
> didn't hear about Scudo before, but it seems it is a LLVM side
> project: https://llvm.org/docs/ScudoHardenedAllocator.html
>
> So perhaps you could build and use it on Linux as shown in
> https://llvm.org/docs/ScudoHardenedAllocator.html#library
>
> Besides a potential bug in the allocator, it might be that the S57
> driver has a memory allocation pattern that doesn't please Scudo.
>
> Assuming that the problematic memory allocations are done using GDAL's
> VSIMalloc() (and not C++ new/delete), then have a look at the various
> #define that you can set at the top of port/cpl_vsisimple.cpp and can
> be used to trace memory allocations
>
> // Uncomment to check consistent usage of VSIMalloc(), VSIRealloc(),
> // VSICalloc(), VSIFree(), VSIStrdup().
> // #define DEBUG_VSIMALLOC
>
> // Uncomment to compute memory usage statistics.
> // DEBUG_VSIMALLOC must also be defined.
> // #define DEBUG_VSIMALLOC_STATS
>
> // Uncomment to print every memory allocation or deallocation.
> // DEBUG_VSIMALLOC must also be defined.
> // #define DEBUG_VSIMALLOC_VERBOSE
>
> // Number of bytes of the malloc/calloc/free that triggers a debug trace.
> // Can be 0 for all allocs.
> #define THRESHOLD_PRINT 10000
>
> Even
>
>
> Le 28/03/2022 à 14:58, Philippe Lelong a écrit :
>>
>> Hi,
>>
>>
>> I am searching for this issue for months now, and cannot find any
>> solution.
>>
>>
>> To make a long story short, we are using GDAL to decode OGR/S57
>> charts for years now. We are facing numerous crashes under Android 11
>> and up if and only if this Android 11 implementation is using SCUDO
>> as a memory allocator (if jemalloc is used no problems). We face this
>> problem with an old GDAL2.1.3 version, so we updated to GDAL 3.4.1
>> but the issue is the same.
>>
>>
>> What I can see is that the memory grows exponentially until no more
>> memory is available and crash, even on systems with huge memory
>> available while an Android device without SCUDO and very limited
>> memory (let's say 4Gb) in the same exact conditions, with the same
>> apk, runs perfectly. The logcat command show this:
>>
>> 03-28 12:40:34.255 4959 5005 W libc : malloc(264196) failed:
>> returning null pointer
>> 03-28 12:40:34.255 4959 5005 W libc : malloc(264196) failed:
>> returning null pointer
>> 03-28 12:40:34.256 4959 5005 W libc : malloc(264196) failed:
>> returning null pointer
>> 03-28 12:40:34.256 4959 5005 W libc : malloc(264196) failed:
>> returning null pointer
>> 03-28 12:40:34.612 630 630 D io_stats: !@ Write_top(KB):
>> kworker/u16:1(32583) 8
>> 03-28 12:40:34.820 4959 5041 I scudo : Scudo ERROR: out of memory
>> trying to allocate 64 bytes
>> 03-28 12:40:34.820 4959 5042 I scudo : Scudo ERROR: out of memory
>> trying to allocate 64 bytes
>> 03-28 12:40:34.820 4959 5033 I scudo : Scudo ERROR: out of memory
>> trying to allocate 64 bytes
>> 03-28 12:40:34.820 4959 5031 I scudo : Scudo ERROR: out of memory
>> trying to allocate 64 bytes
>> 03-28 12:40:34.820 4959 5038 I scudo : Scudo ERROR: out of memory
>> trying to allocate 64 bytes
>> 03-28 12:40:34.820 4959 5040 I scudo : Scudo ERROR: out of memory
>> trying to allocate 64 bytes
>>
>> and then crash
>>
>> Any help on how to debug and eventually fix this would be highly
>> appreciated.
>>
>> Best regards,
>> Philippe from qtVlm development team.
>>
>>
>> _______________________________________________
>> gdal-dev mailing list
>> gdal-dev at lists.osgeo.org
>> https://lists.osgeo.org/mailman/listinfo/gdal-dev
> --
> http://www.spatialys.com
> My software is free, but my time generally not.
>
> _______________________________________________
> gdal-dev mailing list
> gdal-dev at lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/gdal-dev
--
http://www.spatialys.com
My software is free, but my time generally not.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20220402/2c0e3454/attachment-0001.html>
More information about the gdal-dev
mailing list