[gdal-dev] Memory allocation issues on Android 11+ and scudo

Philippe Lelong lelong.ph at meltemus.com
Sat Apr 2 01:17:30 PDT 2022


Hi again,


Still fighting with that one.


I have tried to not run gdal multithreaded, same issue. I have also tried to replace malloc(nSize) with calloc(1, nSize) in VSIMalloc to make sure the memory is initialized, same issue. And many other things... still crashing.


I don't know how to interpret the logs I joined in the previous mail with VSIMalloc etc debugging, do you see anything special in them?


Any other idea I can try to isolate the problem?


Thanks in advance

Philippe.


________________________________
From: gdal-dev <gdal-dev-bounces at lists.osgeo.org> on behalf of Philippe Lelong <lelong.ph at meltemus.com>
Sent: Tuesday, March 29, 2022 7:15 AM
To: gdal-dev at lists.osgeo.org
Subject: Re: [gdal-dev] Memory allocation issues on Android 11+ and scudo


Hi,

I have some results.


First of all note that fprintf(stderr,...)  does nothing on Android since it is redirected to /dev/null. I had to modify cpl_simple.txt in order to use

__android_log_print(ANDROID_LOG_INFO,LOG_TAG, ... instead.


You can find the full logs here: https://www.virtual-winds.org/maitai/gdal_crash.zip


Here is the end of one of them, just before the crash:



03-28 21:02:14.430 20211 20268 E GDAL_VSI: Thread[0x7765183cb0] VSIRealloc(0x7736492fd0, 14768) = 0x7736489630, current_cumul = 22725820, mal+cal-free = 163282
03-28 21:02:14.430 20211 20268 E GDAL_VSI: Thread[0x7765183cb0] VSIRealloc(0x7736489630, 14784) = 0x7736489630, current_cumul = 22725836, mal+cal-free = 163282
03-28 21:02:14.430 20211 20268 E GDAL_VSI: Thread[0x7765183cb0] VSIMalloc(14784) = 0x773645f620, current_cumul = 22740668, mal+cal-free = 163283
03-28 21:02:14.430 20211 20268 E GDAL_VSI: Thread[0x7765183cb0] VSIRealloc(0x773645f620, 14800) = 0x773645f620, current_cumul = 22740684, mal+cal-free = 163284
03-28 21:02:14.430 20211 20268 E GDAL_VSI: Thread[0x7765183cb0] VSIRealloc(0x773645f620, 29568) = 0x7736455c60, current_cumul = 22755452, mal+cal-free = 163284
03-28 21:02:14.430 20211 20268 E GDAL_VSI: Thread[0x7765183cb0] VSIRealloc(0x7736455c60, 29584) = 0x7736455c60, current_cumul = 22755468, mal+cal-free = 163284
03-28 21:02:14.430 20211 20268 E GDAL_VSI: Thread[0x7765183cb0] VSIFree(0x773669f630, (14784 bytes))
03-28 21:02:14.430 20211 20268 E GDAL_VSI: Thread[0x7765183cb0] VSIFree(0x7736489630, (14784 bytes))
03-28 21:02:14.431  1781  5039 D BtGatt.ContextMap: remove() - id: 10
03-28 21:02:14.431  1781  5039 E BtGatt.ContextMap: remove() - removed: 10
03-28 21:02:14.432  1323  1567 D ActivityManager: Received SERVICE intent 0xa7cee6b Key{startService pkg=com.sec.android.app.shealth intent=act=com.samsung.android.app.shealth.tracker.pedometer.PedometerService.HBA cmp=com.sec.android.app.shealth/com.samsung.android.app.shealth.tracker.pedometer.service.PedometerService flags=0x0 u=0} requestCode=0 from uid 1000
03-28 21:02:14.433  1781  3013 I bt_stack: [INFO:gatt_api.cc(1102)] GATT_Deregister gatt_if=10
03-28 21:02:14.434  3224  3224 W [MCFServer]_BleAdapterCallbackManager: onCallbackDied - client was dead, unregister bleAdapter callback appId:17
03-28 21:02:14.435  3224  3224 I [MCFServer]_McfMainController: destroyBleAdapter - appId=17 , mBleAdapterCallbacks size:0
03-28 21:02:14.435  3224  3224 I [MCFServer]_McfMainController: destroyBleAdapter - mBleAdapterCallbacks is empty, destroyBleAdapterManager after 3s
03-28 21:02:14.435  1781  3013 I bt_stack: [INFO:gatt_api.cc(1155)] Initialize tGATT_REG
03-28 21:02:14.438  1781  2610 E BtGatt.GattService: [GSIM LOG]: gsimLogHandler, msg: MESSAGE_SCAN_STOP, appName: com.samsung.android.mcfserver, scannerId: 10, reportDelayMillis=0
03-28 21:02:14.440  1323  1567 V SamsungAlarmManager: setLocked to kernel - W:94823351 / NW:94183790, now=94162423
03-28 21:02:14.441  1323  1323 W Looper  : Drained
03-28 21:02:14.441  1323  3033 W LocationManagerService: onFreezeStateChanged, uid[10400]=false
03-28 21:02:14.441  1323  3033 I PowerManagerService: [PWL] SetWakeLockEnableDisable uid = 10400 , disable= false
03-28 21:02:14.441  1323  3033 I PowerManagerService: [PWL] can not change uid =  10400
03-28 21:02:14.442  1323  1360 V SamsungAlarmManager: setLocked to kernel - W:94823351 / NW:94183790, now=94162425
03-28 21:02:14.444  1323  1546 W system_server: Long monitor contention with owner AlarmManager (1567) at void com.android.server.alarm.AlarmManagerService$AlarmThread.run()(AlarmManagerService.java:5025) waiters=0 in void com.android.server.alarm.AlarmManagerService$DeliveryTracker.alarmComplete(android.os.IBinder) for 296ms
03-28 21:02:14.444   857   857 D Zygote  : Forked child process 21000
03-28 21:02:14.444  1323  3033 W LocationManagerService: onFreezeStateChanged, uid[10301]=false
03-28 21:02:14.445  1323  3033 I PowerManagerService: [PWL] SetWakeLockEnableDisable uid = 10301 , disable= false
03-28 21:02:14.445  1323  1432 I ActivityManager: Start proc 21000:android:drmService/1000 for service {android/com.android.server.DrmEventService}
03-28 21:02:14.445  1323  3033 I PowerManagerService: [PWL] can not change uid =  10301
03-28 21:02:14.445  1323  1431 W BroadcastQueue: Skipping deliver [background] BroadcastRecord{9698798 u-1 android.intent.action.BATTERY_CHANGED} to ReceiverList{8e48bc4 19599 com.facebook.orca/10400/u0 remote:393b6d7}: process gone or crashing
03-28 21:02:14.445  1323  1431 W BroadcastQueue: Skipping deliver [background] BroadcastRecord{9698798 u-1 android.intent.action.BATTERY_CHANGED} to ReceiverList{687dc5e 19599 com.facebook.orca/10400/u0 remote:b49e199}: process gone or crashing
03-28 21:02:14.446  1323  1431 W BroadcastQueue: Skipping deliver [background] BroadcastRecord{cf3bf1 u-1 android.intent.action.BATTERY_CHANGED} to ReceiverList{26d4e3d 19145 com.facebook.katana/10301/u0 remote:87e8394}: process gone or crashing
03-28 21:02:14.446  1323  3033 W LocationManagerService: onFreezeStateChanged, uid[10382]=false
03-28 21:02:14.446  1323  3033 I PowerManagerService: [PWL] SetWakeLockEnableDisable uid = 10382 , disable= false
03-28 21:02:14.446  1323  3033 I PowerManagerService: [PWL] can not change uid =  10382
03-28 21:02:14.447  1809  1809 D QS      : setQSExpansion 0.0 -122.85
03-28 21:02:14.449  8549  8549 I PedometerService: onStartCommand  true, true, true, 1648400236406, 1648400236842, false, false
03-28 21:02:14.451  1809  1809 V SecQSFragmentAnimatorBase: setQsExpansionPosition 0.0
03-28 21:02:14.451  1809  1809 V QsExpandAnimator: setQsExpansionPosition 0.0 0
03-28 21:02:14.451  1323  1431 W BroadcastQueue: Skipping deliver [background] BroadcastRecord{cf3bf1 u-1 android.intent.action.BATTERY_CHANGED} to ReceiverList{86abcc6 19145 com.facebook.katana/10301/u0 remote:8f3e2a1}: process gone or crashing
03-28 21:02:14.452  1323  1360 D SamsungAlarmManager: setInexact (T:3/F:0/AC:false) 20220329T000214 now=94162435 - CU:10251/CP:2699/OP:PendingIntent{9e623d6: PendingIntentRecord{650d89a com.google.android.gms/com.google.android.gms.tron broadcastIntent}}
03-28 21:02:14.452  1323  1360 V SamsungAlarmManager: setLocked to kernel - W:94823351 / NW:94183790, now=94162435
03-28 21:02:14.457  1323  2605 D SemContextService: lock : requestToUpdate
03-28 21:02:14.458  1323  2605 D SemContext.CaeProvider.SensorStatusCheckImpl: Sensor Check Event is null!!
03-28 21:02:14.458  1323  2605 D SemContextService:     .requestToUpdate() : service = Sensor Status Check
03-28 21:02:14.458  1323  2605 D SemContextService: unlock : requestToUpdate
03-28 21:02:14.464  1809  1809 D QS      : setQSExpansion 0.0 -122.85
03-28 21:02:14.466  8549 31023 I SHEALTH#WI#WearableConnectionMonitor: (print) : getConnectedWearableDeviceList(), size : 1
03-28 21:02:14.466 20211 20264 E GDAL_VSI: Thread[0x776957bcb0] VSIRealloc(0x78ab48f090, 13504) = 0x77341bbb20, current_cumul = 22743760, mal+cal-free = 163385
03-28 21:02:14.466 20211 20264 E GDAL_VSI: Thread[0x776957bcb0] VSIRealloc(0x77341bbb20, 13520) = 0x77341bbb20, current_cumul = 22743776, mal+cal-free = 163385
03-28 21:02:14.470 21000 21000 E android:drmSer: Not starting debugger since process cannot load the jdwp agent.
03-28 21:02:14.471 21000 21000 E USNET   : USNET: appName: android:drmService
03-28 21:02:14.471 21000 21000 D ProcessState: Binder ioctl to enable oneway spam detection failed: Invalid argument
03-28 21:02:14.473   996  1051 I heimdall: insert_task_to_group:64, insert tgid 21000 to group com.google.process.gapps, ret = 0
03-28 21:02:14.476 21000 21000 D ActivityThread: setConscryptValidator
03-28 21:02:14.476 21000 21000 D ActivityThread: setConscryptValidator - put
03-28 21:02:14.479  1323  2605 I ActivityManager: DSS OFF for android
03-28 21:02:14.484  1323  2605 D ActivityManager: attachApplicationLocked() app=ProcessRecord{60e4cac 21000:android:drmService/1000} app.isolatedEntryPoint=null instr2=null
03-28 21:02:14.492  1809  1809 W LooperSlow: RunCallback: type=3, action=android.view.ViewRootImpl$TraversalRunnable at d06ee61, token=null, latencyMillis=918, dur=91ms
03-28 21:02:14.494  1809  1809 D IndicatorGardenInputProperty: updateRotation() prv:-1 >> new:0
03-28 21:02:14.495  1809  1809 D DeviceState: getDeviceResolutionPixelSize - currentDensity = 450 deviceDensity = 450 initialDisplaySizeFactor = 1440 currentDisplaySizeFactor = 1080 initialDisplayDensity = 600 proportionalDensity = 450 proportionalPixel = 72
03-28 21:02:14.495  1809  1809 D DeviceState: getDeviceResolutionPixelSize - currentDensity = 450 deviceDensity = 450 initialDisplaySizeFactor = 1440 currentDisplaySizeFactor = 1080 initialDisplayDensity = 600 proportionalDensity = 450 proportionalPixel = 27
03-28 21:02:14.495  1809  1809 D DeviceState: getDeviceResolutionPixelSize - currentDensity = 450 deviceDensity = 450 initialDisplaySizeFactor = 1440 currentDisplaySizeFactor = 1080 initialDisplayDensity = 600 proportionalDensity = 450 proportionalPixel = 16
03-28 21:02:14.495  1809  1809 E IndicatorGardenAlgorithmBasicCutout: NOT MATCH !!!! resourceHeight:74, cutoutHeight:75 ([IndicatorGardenInputProperty]  Rotation(0-0,90-1,180-2,270-3)0, Density:2.8125, ScreenWidthSize:1080, CoverSidePadding:0, mIndicatorGardenCenterOffset:12, mCameraCutoutCropSize:0, mGardenPaddingStart:23, mIndicatorCornerPadding:72, mCameraSidePadding:27, mCameraTopMargin:16, DpCutout:DisplayCutout{insets=Rect(0, 75 - 0, 0) waterfall=Insets{left=0, top=0, right=0, bottom=0} boundingRect={Bounds=[Rect(0, 0 - 0, 0), Rect(511, 0 - 569, 75), Rect(0, 0 - 0, 0), Rect(0, 0 - 0, 0)]} cutoutPathParserInfo={CutoutPathParserInfo{displayWidth=1080 displayHeight=2400 density={2.8125} cutoutSpec={M 0, 0 H -10.4 V 26.66666666666667 H 10.4 V 0 H 0 Z @dp} rotation={0} scale={1.0}}}})
03-28 21:02:14.495  1809  1809 E IndicatorGardenAlgorithmBasicCutout: NOT MATCH !!!! resourceHeight:74, cutoutHeight:75 ([IndicatorGardenInputProperty]  Rotation(0-0,90-1,180-2,270-3)0, Density:2.8125, ScreenWidthSize:1080, CoverSidePadding:0, mIndicatorGardenCenterOffset:12, mCameraCutoutCropSize:0, mGardenPaddingStart:23, mIndicatorCornerPadding:72, mCameraSidePadding:27, mCameraTopMargin:16, DpCutout:DisplayCutout{insets=Rect(0, 75 - 0, 0) waterfall=Insets{left=0, top=0, right=0, bottom=0} boundingRect={Bounds=[Rect(0, 0 - 0, 0), Rect(511, 0 - 569, 75), Rect(0, 0 - 0, 0), Rect(0, 0 - 0, 0)]} cutoutPathParserInfo={CutoutPathParserInfo{displayWidth=1080 displayHeight=2400 density={2.8125} cutoutSpec={M 0, 0 H -10.4 V 26.66666666666667 H 10.4 V 0 H 0 Z @dp} rotation={0} scale={1.0}}}})
03-28 21:02:14.495 21000 21000 D ActivityThread: handleBindApplication()++ app=android:drmService
03-28 21:02:14.496 21000 21000 D CompatibilityChangeReporter: Compat change id reported: 171979766; UID 1000; state: ENABLED
03-28 21:02:14.505  1809  1809 D SystemUIService: SYSUI_RAM_OPTIMIZATION onTrimMemory=15
03-28 21:02:14.506   583   583 I lmkd    : 2(delay),0(swap), 0(freelimit) memory pressure events were skipped after a kill!
03-28 21:02:14.506  1809  1809 D StatusBar: SYSUI_RAM_OPTIMIZATION onTrimMemory=15
03-28 21:02:14.506  1809  1809 D SystemUIService: onTrimMemory : 15
03-28 21:02:14.506   583   583 I lmkd    : cached 0, sandbox(not0) 0
03-28 21:02:14.506  1809  1809 D SystemUIService: Last Info is 03-28 21:00:08.559. It still remains until reset time. So skip this.
03-28 21:02:14.506  1809  1809 D SystemUIService: SYSUI_RAM_OPTIMIZATION onTrimMemory=15
03-28 21:02:14.506  1809  1809 D StatusBar: SYSUI_RAM_OPTIMIZATION onTrimMemory=15
03-28 21:02:14.506  1809  1809 D SystemUIService: onTrimMemory : 15
03-28 21:02:14.507  1809  1809 D SystemUIService: Last Info is 03-28 21:00:08.559. It still remains until reset time. So skip this.
03-28 21:02:14.507  1809  1809 W Looper  : Slow dispatch took 106ms main h=android.view.Choreographer$FrameHandler c=android.view.Choreographer$FrameDisplayEventReceiver at 15dac12 m=0
03-28 21:02:14.510   583   583 E libprocessgroup: set_timerslack_ns write failed: No such process
03-28 21:02:14.511   583   583 I lmkd    : Reclaim 'android.process.acore' (27861), uid 10071, oom_score_adj 850, state 99 to free 33916kB rss, 56816kB swap; reason: low watermark is breached and swap is low (1848484kB < 838860kB)
03-28 21:02:14.521 20211 20264 E GDAL_VSI: Thread[0x776957bcb0] VSIFree(0x77341bbb20, (13520 bytes))
03-28 21:02:14.533   857   857 I Zygote  : Process 27861 exited due to signal 9 (Killed)
03-28 21:02:14.533  1323  5047 D InputMethodManagerService: removeClient
03-28 21:02:14.533   583   583 I lmkd    : cached 0, sandbox(not0) 0
03-28 21:02:14.535   583   583 E libprocessgroup: set_timerslack_ns write failed: No such process
03-28 21:02:14.537  1323  2310 I ActivityManager: Process android.process.acore (pid 27861) has died: picked CEM (290,316)
03-28 21:02:14.537  1323  1433 I libprocessgroup: Successfully killed process cgroup uid 10071 pid 27861 in 0ms
03-28 21:02:14.538   583   583 I lmkd    : Reclaim 'com.samsung.cmh:CMH' (28089), uid 5004, oom_score_adj 850, state 99 to free 29444kB rss, 53292kB swap; reason: low watermark is breached and swap is low (1863132kB < 838860kB)
03-28 21:02:14.539  1781  1781 D HidDeviceService: handleMessage(): msg.what=8
03-28 21:02:14.540  1809  1809 W LooperSlow: RunCallback: type=3, action=android.view.ViewRootImpl$TraversalRunnable at fe916b9, token=null, latencyMillis=31, dur=31ms
03-28 21:02:14.540  1809  1809 W Looper  : Slow dispatch took 31ms main h=android.view.Choreographer$FrameHandler c=android.view.Choreographer$FrameDisplayEventReceiver at 15dac12 m=0
03-28 21:02:14.541  1809  1809 W Looper  : Drained
03-28 21:02:14.557 20211 20264 E GDAL_VSI: Thread[0x776957bcb0] VSIRealloc(0x789d45b450, 21280) = 0x772f097cc0, current_cumul = 22758320, mal+cal-free = 163504
03-28 21:02:14.557 20211 20264 E GDAL_VSI: Thread[0x776957bcb0] VSIRealloc(0x772f097cc0, 21296) = 0x772f097cc0, current_cumul = 22758336, mal+cal-free = 163504
03-28 21:02:14.558 20211 20269 E GDAL_VSI: Thread[0x7764085cb0] VSIFree(0x7a89741610, (55312 bytes))
03-28 21:02:14.566   857   857 I Zygote  : Process 28089 exited due to signal 9 (Killed)
03-28 21:02:14.566  1323  1553 I ActivityManager: Process com.samsung.cmh:CMH (pid 28089) has died: picked CEM (277,320)
03-28 21:02:14.566   583   583 I lmkd    : cached 0, sandbox(not0) 0
03-28 21:02:14.566 20211 20262 E GDAL_VSI: Thread[0x7770e3dcb0] VSIRealloc(0x7a3f23d980, 57120) = 0x7a3f23d980, current_cumul = 22716708, mal+cal-free = 163501
03-28 21:02:14.566  1323  1433 I libprocessgroup: Successfully killed process cgroup uid 5004 pid 28089 in 0ms
03-28 21:02:14.568  1781  1781 D HidDeviceService: handleMessage(): msg.what=8
03-28 21:02:14.575   583   583 I lmkd    : Reclaim 'com.google.android.gms' (18299), uid 10251, oom_score_adj 800, state 10 to free 41892kB rss, 50312kB swap; reason: low watermark is breached and swap is low (1871592kB < 838860kB)
03-28 21:02:14.588 20983 20983 V GraphicsEnvironment: ANGLE Developer option for 'com.google.android.apps.turbo' set to: 'default'
03-28 21:02:14.594  1809  3054 I OpenGLRenderer: Davey! duration=989ms; Flags=0, FrameTimelineVsyncId=1355581, IntendedVsync=50486826840932, Vsync=50487676840898, InputEventId=0, HandleInputStart=50487682229734, AnimationStart=50487682230811, PerformTraversalsStart=50487682231618, DrawStart=50487777157695, FrameDeadline=50486843507598, FrameInterval=50487682147349, FrameStartTime=16666666, SyncQueued=50487779379772, SyncStart=50487780687541, IssueDrawCommandsStart=50487786418772, SwapBuffers=50487808932657, FrameCompleted=50487817955695, DequeueBufferDuration=1428731, QueueBufferDuration=4194269, GpuCompleted=50487811641964, SwapBuffersCompleted=50487817955695, DisplayPresentTime=0,
03-28 21:02:14.608  1323  2605 D ConnectivityService: ConnectivityService NetworkRequestInfo binderDied(uid/pid:10251/18299, [NetworkRequest [ LISTEN id=3417, [ Transports: WIFI Capabilities: NOT_RESTRICTED&TRUSTED&NOT_VPN&NOT_VCN_MANAGED Uid: 10251 RequestorUid: 10251 RequestorPkg: com.google.android.gms UnderlyingNetworks: Null] ]], android.os.BinderProxy at f2b50f3)
03-28 21:02:14.608   583   583 I lmkd    : cached 0, sandbox(not0) 0
03-28 21:02:14.609   857   857 I Zygote  : Process 18299 exited due to signal 9 (Killed)
03-28 21:02:14.610  1323  2457 D ConnectivityService: ConnectivityService NetworkRequestInfo binderDied(uid/pid:10251/18299, [NetworkRequest [ LISTEN id=3418, [ Transports: WIFI Capabilities: NOT_RESTRICTED&TRUSTED&NOT_VPN&NOT_VCN_MANAGED Uid: 10251 RequestorUid: 10251 RequestorPkg: com.google.android.gms UnderlyingNetworks: Null] ]], android.os.BinderProxy at 70db1b0)
03-28 21:02:14.610  1323  2605 I ActivityManager: Process com.google.android.gms (pid 18299) has died: svc SVC (283,318)
03-28 21:02:14.611  1323  1433 I libprocessgroup: Successfully killed process cgroup uid 10251 pid 18299 in 0ms
03-28 21:02:14.611  1323  2605 W ActivityManager: Scheduling restart of crashed service com.google.android.gms/.cast.service.CastPersistentService in 50492ms for start-requested
03-28 21:02:14.613  1323  5045 E WifiMulticastLockManager: Multicaster binderDied
03-28 21:02:14.615   583   583 E libprocessgroup: set_timerslack_ns write failed: No such process
03-28 21:02:14.616 20211 20267 W libc    : malloc(40) failed: returning null pointer
03-28 21:02:14.616 20211 20263 W libc    : malloc(40) failed: returning null pointer
03-28 21:02:14.616 20211 20266 W libc    : malloc(34) failed: returning null pointer
03-28 21:02:14.616 20211 20269 W libc    : malloc(72) failed: returning null pointer
03-28 21:02:14.616 20211 20268 W libc    : malloc(32) failed: returning null pointer
03-28 21:02:14.616 20211 20262 W libc    : malloc(68) failed: returning null pointer
03-28 21:02:14.616 20211 20264 W libc    : malloc(32) failed: returning null pointer
03-28 21:02:14.617 20211 20265 W libc    : malloc(32) failed: returning null pointer
03-28 21:02:14.617 20211 20269 W libc    : malloc(128) failed: returning null pointer
--------- beginning of crash
03-28 21:02:14.617 20211 20262 F libc    : Fatal signal 6 (SIGABRT), code -1 (SI_QUEUE) in tid 20262 (Thread (pooled)), pid 20211 (.meltemus.qtvlm)
03-28 21:02:14.617 20211 20264 W libc    : malloc(128) failed: returning null pointer

I hope you can understand something, I don't.

Best regards and thank


________________________________
From: gdal-dev <gdal-dev-bounces at lists.osgeo.org> on behalf of Philippe Lelong <lelong.ph at meltemus.com>
Sent: Monday, March 28, 2022 3:50 PM
To: gdal-dev at lists.osgeo.org
Subject: Re: [gdal-dev] Memory allocation issues on Android 11+ and scudo


Thanks Even and Greg for your fast replies


The app and GDAL are built with Android NDK 21 (I have tried NDK 30 and 31) and is targeting SDK 30. High memory Android device is 10Gb or even 14Gb with RAM PLUS. I can add that the app itself is based on Qt 5.15.8 and runs fine on Windows, Linux, Raspberry 2/3/4 32bits or 64bits, MacOS, iOS and Android without Scudo. It is heavily multithreaded including during GDAL calls.


I can also add that I have tried a 32bits build (armv7) running on these armV8 devices, same problem, although the messages in logcat are a bit different. I can also say that I am 99.9% sure this is triggered by GDAL, because our same exactly S52/57 module reading other formats without GDAL does not crash.


I have seen these debug defines in cpl_simple.cpp. I will enable them, rebuild and report here. I don't have a scudo device apart from a emulated one but it is x86-64 and it does not seem to behave exactly the same as the users' ARM64 bits (Samsung S21). My feeling is that it is connected to ARM 64 architecture.


Thanks again, more later hopefully.

Philippe.


________________________________
From: Even Rouault <even.rouault at spatialys.com>
Sent: Monday, March 28, 2022 3:24 PM
To: Philippe Lelong; gdal-dev at lists.osgeo.org
Subject: Re: [gdal-dev] Memory allocation issues on Android 11+ and scudo


Hi,

didn't hear about Scudo before, but it seems it is a LLVM side project: https://llvm.org/docs/ScudoHardenedAllocator.html

So perhaps you could build and use it on Linux as shown in https://llvm.org/docs/ScudoHardenedAllocator.html#library

Besides a potential bug in the allocator, it might be that the S57 driver has a memory allocation pattern that doesn't please Scudo.

Assuming that the problematic memory allocations are done using GDAL's VSIMalloc() (and not C++ new/delete), then have a look at the various #define that you can set at the top of port/cpl_vsisimple.cpp and can be used to trace memory allocations

// Uncomment to check consistent usage of VSIMalloc(), VSIRealloc(),
// VSICalloc(), VSIFree(), VSIStrdup().
// #define DEBUG_VSIMALLOC

// Uncomment to compute memory usage statistics.
// DEBUG_VSIMALLOC must also be defined.
// #define DEBUG_VSIMALLOC_STATS

// Uncomment to print every memory allocation or deallocation.
// DEBUG_VSIMALLOC must also be defined.
// #define DEBUG_VSIMALLOC_VERBOSE

// Number of bytes of the malloc/calloc/free that triggers a debug trace.
// Can be 0 for all allocs.
#define THRESHOLD_PRINT 10000

Even



Le 28/03/2022 à 14:58, Philippe Lelong a écrit :

Hi,


I am searching for this issue for months now, and cannot find any solution.

To make a long story short, we are using GDAL to decode OGR/S57 charts for years now. We are facing numerous crashes under Android 11 and up if and only if this Android 11 implementation is using SCUDO as a memory allocator (if jemalloc is used no problems). We face this problem with an old GDAL2.1.3 version, so we updated to GDAL 3.4.1 but the issue is the same.


What I can see is that the memory grows exponentially until no more memory is available and crash, even on systems with huge memory available while an Android device without SCUDO and very limited memory (let's say 4Gb) in the same exact conditions, with the same apk, runs perfectly. The logcat command show this:



03-28 12:40:34.255  4959  5005 W libc    : malloc(264196) failed: returning null pointer
03-28 12:40:34.255  4959  5005 W libc    : malloc(264196) failed: returning null pointer
03-28 12:40:34.256  4959  5005 W libc    : malloc(264196) failed: returning null pointer
03-28 12:40:34.256  4959  5005 W libc    : malloc(264196) failed: returning null pointer
03-28 12:40:34.612   630   630 D io_stats: !@ Write_top(KB): kworker/u16:1(32583) 8
03-28 12:40:34.820  4959  5041 I scudo   : Scudo ERROR: out of memory trying to allocate 64 bytes
03-28 12:40:34.820  4959  5042 I scudo   : Scudo ERROR: out of memory trying to allocate 64 bytes
03-28 12:40:34.820  4959  5033 I scudo   : Scudo ERROR: out of memory trying to allocate 64 bytes
03-28 12:40:34.820  4959  5031 I scudo   : Scudo ERROR: out of memory trying to allocate 64 bytes
03-28 12:40:34.820  4959  5038 I scudo   : Scudo ERROR: out of memory trying to allocate 64 bytes
03-28 12:40:34.820  4959  5040 I scudo   : Scudo ERROR: out of memory trying to allocate 64 bytes

and then crash

Any help on how to debug and eventually fix this would be highly appreciated.

Best regards,
Philippe from qtVlm development team.




_______________________________________________
gdal-dev mailing list
gdal-dev at lists.osgeo.org<mailto:gdal-dev at lists.osgeo.org>
https://lists.osgeo.org/mailman/listinfo/gdal-dev


--
http://www.spatialys.com
My software is free, but my time generally not.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20220402/144862be/attachment-0001.html>


More information about the gdal-dev mailing list