[mapserver-users] mapcache_seed segfault when using threading [SEC=UNCLASSIFIED]

[thomas bonfort]

Matthew,
What would help is if you could provide a backtrace from the segfault:

- reconfigure with --enable-debug / recompile / reinstall
- gdb mapcache_seed
  - run [previous args: -n 4 -c /path/to/mapcache.xml etc...]
  - wait till segfault
  - bt

thanks,
thomas

On Wed, Oct 10, 2012 at 8:14 AM, Matthew Doyle <M.Doyle at bom.gov.au> wrote:
> Hi Thomas & MapCache users,
>
> A little background: we are developing a weather forecasting tool which
> provides a visual representation of a quite a large dataset (NetCDF grid, 7
> days of data every three hours). As such, we have many diferent layers and a
> very large number of tiles which require seeding multiple times per day.
>
> Based on very impressive seeding results from tests carried out on on RHEL 5
> (a VM box with 8 cores), seeding using 8 threads, and encountering no issues
> during these tests, the decision was made to purchase the best hardware
> available in order to seed our tiles as quickly as possible when going
> operational.
>
>
> The issue: mapcache_seed is halting with "segmentation fault (core dumped)"
> on our new operational machine when using any number of threads > 1. The
> higher the number of threads, the more likely it is to segfault. The issue
> appears during the initialisation phase just prior to seeding as once the
> seeding run is initialised it will complete successfully, but if the
> segfault occurs it will occur and exit almost immediately after execution.
>
> I will try to provide as much information as possible, please let me know
> what would be useful, perhaps the core dump ? or a simple test case
> (MapServer mapfile / data file / mapcache xml / seeding parameters) in the
> hope that you might be able to reproduce?
>
> Hardware:
>
> Power Edge R820, 4-socket, 2U blade server
> Populated with four Intel Xeon E5 processors
> Processor E5-4650L - 20M Cache, 2.60 GHz, 8.00 GT/s), Clock Speed2.6 GHz.
> Max
> Turbo Frequency 3.1 GHz,
> Each processor has 8 (microprocessor) cores,
> Supporting 16 threads per core with hyper threading,
> A total of 64 threads, and
> 300GB RAM. (48 DIMMS (upgradable to 1.5 TB of memory))
>
> OS:
>
> Red Hat Enterprise Linux Server release 6.3 (Santiago)
>
> Software:
>
> Apache Server version: Apache/2.2.15 (Unix)
>
> mod_wsgi
> mod_mapcache
>
> MapServer 6.0.3
> MapCache (current devel 6.2.0beta2)
> Gdal 1.8.1
> Python 2.6.6
>
> Python Mapscript
>
> Perl 5.10.1
>
> Perl Mapscript
>
> Postgres 8.4.12/PostGIS 1.5.3
>
> Our WMS service is a very basic Python MapScript wrapper via mod_wsgi, but i
> have also tried standard CGI Python and CGI Perl wrappers with the same
> results.
>
> We are using non-linear resolution steps to avoid seeding too many tiles.
> Perhaps that is a problem with depth seeding?
> We are using a metatile area of 8 x 8.
>
> Attempting to seed using any number of threads will cause a segfault
> (usually any number higher than 4, but i have seen it fail with less). It's
> worth mentioning that if n threads is set to 1, then the seeding process
> executes and completes without any issues. Any light you can shed on this
> situation and any suggestions are greatly appreciated.
>
> Best regards,
> Matt