[gdal-dev] Call for discussion on "RFC 45: GDAL datasets and raster bands as virtual memory mappings"

[Even Rouault]

Le jeudi 19 décembre 2013 04:41:18, Trent Piepho a écrit :
> Do you see page file activity?  If you look at /proc/pid/smaps, you
> should be able to see the actual status of the mapping of your data
> file.  Probably it is consuming a large number of pages of RAM, but
> also there should be zero pages written to swap.  All clean private or
> clean shared, zero anonymous and zero swap.

7f8e6f75d000-7f9334df1000 r--p 00000000 08:06 11645756                   
/home/even/Téléchargements/eudem_dem_4258_europe.tif
Size:           20011600 kB
Rss:             2943372 kB
Pss:             2943372 kB
Shared_Clean:          0 kB
Shared_Dirty:          0 kB
Private_Clean:   2943372 kB
Private_Dirty:         0 kB
Referenced:       950596 kB
Swap:                  0 kB
KernelPageSize:        4 kB
MMUPageSize:           4 kB

So yes, you are right. I've noticed the 'Referenced' values tends to fluctuate 
a lot during execution, while other values remain stable after some point.

> 
> I think the system unresponsiveness is probably do to I/O scheduling.
> You're process has queued a lot of I/O reads and everything else has
> to wait in the queue.  So all other I/O sees huge latencies.

Yes, that's a likely clause. I've just tried to do a 
madvise(..,..,MADV_RANDOM) on the whole mapping just after mmap(), and it 
seems to increase the system responsiveness in a noticeable way (since I can 
write this email while the program is running !). Of course, the throughput of 
the test program has reduced significantly (which is logical since agressive 
I/O read-ahead must be disabled).

> 
> And too, a 20 GB mapping is probably thrashing the TLB.  Do huge pages
> actually get used?

No, they don't. My understanding and previous attempts is that mmap() needs an 
explicit flag for that, and other tunings at OS level.

> On the embedded systems I'm more intimately
> familiar with, only normal 4k pages are used by user processes.  Huge
> TLBs are more of a special case that can be used by the kernel for
> things like frame buffer mappings and SoC register windows.
> 


-- 
Geospatial professional services
http://even.rouault.free.fr/services.html